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BY THE COMPTROLLER GENERAL 




OF THE UNITED STATES 



CETA Demonstration Provid 
Lessons On Implementing 
Youth Programs 



The Youth Incentive Entitlement Pilot Proj- 
ects was designed to test whether guaranteed 
part-time jobs during the school year and full- 
time jobs in the summer would induce eco- 
nomically disadvantaged youths to remain in 
or return tc school. GAO found that most 
worksites provided adequate experiences. 
Others, however, had problems in development 
of good work habits, in supervision, and in 
monitoring and couseling. Also many sponsors 
experienced operational problems. 

The Department of Labor and its contractor 
had to intervene frequently in program oper- 
ations. This raised the questions of wnether the 
demonstration was a fair test of the guaranteed 
job concept and whether sponsors are capable 
of implementing complex programs. 

GAO recommends that, if the Congressdecides 
to expand the Pilot Projects or undertake a 
similar program, it do so gradually to allow 
sponsors time to develop the capacity to prop- 
erly implement the program. 
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To the President of uhe Senate and the 
Speaker of the House of Representatives 

This report dtiscribes youths' work experiences in one of 
the largest demonstration programs: the Federal Government has 
ever attempted in the field of youth employment. In addi- 
tion, it describes serious operational problems encountered 
in implementing the demonstration and questions whether the 
demonstration was conducted in a real-world environment. We 
made this review tu give the Congress information to help it 
develop a sound long-range strategy for dealing with youth 
unemployment: . 

We are sending copies of this report to the Director, 
Office of Management and Rudget, and to the Secretary of 
Labor * ^ A 




Comptroller General 
of the United States 
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COMPTROLLER GENERAL'S 
REPORT TO THE CONGRESS 



CETA DEMONSTRATION PROVIDES 
LESSONS ON IMPLEMENTING 
YOUTH PROGRAMS 



DIGEST 

The Department of Labor's Youth Incentive 

Entitlement Pilot Projects (Entitlement) 

is a $222 million demonstration designed to 

— test whether a guaranteed jobs program 
v/ould induce economically disadvantaged 
youths to remain in or return tc school 
and 

— provide jobs to reduce the high unemploy- 
ment rate among these youths . 

Labor^ while retaining overall responsibility 
for the program, contracted with the Manpower 
Demonstration Research Corporation for the 
research design, management, and evaluation 
of the demonstration. (See pp. 2 to 4 • ) 

GAO found that: 

— Most worksites provided adequcite work ex- 
periences, but others had problems in job 
quality, supervision, job development, 
counseling , and worksite monitoring . 
(See pp. 10 to 24. ) 

— Many sponsors experienced serious operational 
problems implemenL^ing or complying with pro- 
gram requirements . (See pp. 25 to 38 . ) 

— Labor and the Corporation frequently inter- 
vened in sponsor operations, thus raising the 
questions of whether the demonstration was a 
fair test of the guaranteed job concept and 
whether sponsors are capable of implement- 
ing complex programs such as Entitlement . 
(See pp. 47 to 51 . ) 
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RECOMMENDATION TO THE CONGRESS 



If Zrtitleraent is to be extended or used as 
the basis for future youth initiatives, the 
Congress should recognize the conditions 
under which the demonstration was imple- 
mented, certain competitively selected 
sponsors receiving intensive oversight had 
significant operational problems. If the 
Entitlement concept is to be extended, or 
expanded to other sponsors or a similar 
program adopted, GAO recommends that the 
Congress make sure it is implemented 
gradually to allow sponsors time to de- 
velop management capacity. (See p. 53.) 

MOST WORKSITES PROVIDED 
ADEQUATE WORK EXPERIENCES 

In evaluating the quality of the work 
experiences, GAO considered whether par- 
ticipants exhibiv-?d good work habits, 
received enough useful work, and were 
adequately supervised. GAO estimates that 
81 percent of the worksites fostered good 
work habit development, 73 percent provided 
enough useful work, and 74 percent provided 
adequate supervision. However, collectively, 
about 2,100 (40 percent) of the 5,200 work- 
sites lacked one or more of the elements 
above. (See pp. 10 to 19.) 

Most of the problems at worksitss are 
correctable. Sites evaluated, as inade- 
quate generally 

— had too many /ouths assigned for the amount 
of work; 

did not adhere to time and attendance pro- 
cedures ; 

— had work of marginal value to the employer, 
the participant, or the community; 

did not enforce work-hour rules and allowed 

youths to be paid for absences; or 




e 



— had unqualified supervisors. (See 
pp. 10 to 19. ) 

Some sponsors had difficulty developing 
and sustaining a sufficient number of jobs 
in the right locations, particularly during 
t' .e school year. Transportation, school 
scheduling, and administrative problems con- 
tributed to this situation. (See pp. 19 
to 21 . ) 

Spo^^vSors placed a low priority on counsel- 
ing. Most ccanselors spent more time pick- 
ing up time cards and delivering paychecks 
than counseling youths. (See pp. 21 and 22.) 

Most sponsors did lictle worksite monitoring 
during the first year of the demonstration. 
Monitoring increa ,ed during the second year, 
but some problems that should have been 
obvious were not always identified. Others, 
although identified, were not always cor- 
rected. (See pp. 22 to 24.) 

OPERATIONAL PROBLEMS HAMPERED 
PROGRAM PERFORMANCE 

Many sponsors had persistent operational 
problems. Some legislative and program 
requirements were largely ignored; others 
y^ere too difficult to accomplish. Some 
sponsors rapidly expanded small core staffs 
with inexperienced personnel. Staff turn- 
over was high. Ineffective job assignment, 
payrol ^ , and management information systems 
contributed to major reorganization at 
some sponsors. (See pp. 25 to 28.) 

Through September 1979, Entitlement cost 
about $4,600 a year per participant. Pro- 
gram management costs represent about 37 
percent of this amount. Rapid program 
startup combined with unrealistic enroll- 
ment projections and larger than needed 
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staffs initially caused high program 
management/participant cost ratios. Less 
than anticipated enrollment was a major 
cause ox :he problem. (See pp. 28 to 31*) 

One of Entitlement's major features was that 
a youth's job guarantee was conditional on 
satisfactory school attendance and perform- 
ance. This feature never received a fair 
test often due to administrative problems 
and a desire not to reduce already low en- 
rollments. As of December 1979, only 4 per- 
cent of 34^000 youths terminated were re- 
leased for unsatisfactory school performance. 
GAO's review showed that far more youths 
were not meeting academic or attendance 
standards. (See pp. 31 to 33.) 

Sponsors were required to establish relation- 
ships with other groups which could help 
them serve youths. These included school 
systems , community-based organizations , and 
other organizations serving youths. The 
results were mixed. Some sound relationships 
were developed; others deteriorated badly 
over time. (See pp. 33 to 35.' 

One of Entitlement's innovative features was 
the authorization of subsidies up to 100 per- 
cent for salaries paid to participants as an 
inducement to private employers for providing 
youths with jobs and for testing various 
subsidy levels. Many private sector jobs 
were with employers that normally hire young 
workers . GAO recognizes some substitution 
is inevitable in a job creation program 
as large as Entitlement? however, where 
substitution occurs, the job creation impact 
is minimized. (See pp. 35 to 38.) 

INTENSIVE OVERSIGHT REQUIRED 

To respond to questions raised by the Congress, 
Labor engaged the Corporation which designed 
a three-part research program to analyze 

8 



Entitlement's impact, implement.ation, and 
cost. (See pp. 39 to 42.) 

Despite intensive efforts by the Corporation 
and its consultants, data problems persisted 
from the onset of the demonstration. The 
Corporation is confident, however, that its 
cleanup efforts have resulted in reliable 
research data. (See pp. 43 to 47.) 

The Corporation had to intercede in program 
operations at many locations. Significant 
improvements were made at some locations 
which might not have occurred in a normal 
program environment. GAG believes that Labor 
and the Corporation, by providing intensive 
oversight and technical assistance at a level 
not normally accorded these type?: of programs, 
were in a unique position to influence pro- 
gram results- At several locations, this as- 
sistance greatly improved program operations; 
at others, this special attention salvaged 
marginal operations. These actions are in^ 
dicative of the degree of oversight necessary 
to maintain quality youth programs, but one 
can only speculate as to the fate of the En- 
titlement demonstration without the oversight 
it received. (See pp. 47 to 51.) 

RECOMMENDATIONS TO THE 
SECR E TARY OF LABOR 

If Entitlement is extended or expanded, or 
a similar program is adopted, GAO recommends 
that the Secretary 

— require sponsors to strictly enfore reason- 
able attendance and academic standards 
thereby enhancing the program's credi- 
bility and effectiveness for ^oth partici- 
pants and the general public; 

— increase efforts to promote cooperative 
relationships between sponsors , schools , 
and other organizations serving youths; 
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— help sponsors upgrade worksite supervi- 
sion, job development, participant coun- 
seling, and monitoring; 

— create, implement, and enforce adequate 
controls to prevent substitution; and 

--establish and enforce requirements that 
limit program management costs. (See 
p. 54. ) 

CORPORATION AND SPONSOR COMMENTS 

GAG requested comments from Labor, the 
Corporation, and the 11 sponsors included 
in the review. The Corporation and five 
sponsors submitted comments- (See pp. 66 
to 80.) The Corporation and the Baltimore 
sponsor disagreed with GAO's approach to 
this study- GAO's detailed analysis of 
Corporation and sponsor concerns starts 
on page 54. After the 30-day comment per- 
iod provided by Public Law 96-226 expired. 
Labor submitted comments which concurred 
with all of GAO*s recommendations to the 
Secretary • Labor's comments, which are in- 
cluded as appendix X, have not been ana- 
lyzed in this report. 
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CHAPTER 1 

YOUTH UNEMPLOYMENT — A PERSISTENT 

PROBLEM THAT HAS RESISTED SOLUTION 

Youfh unemployment has been a persistent national prob- 
lem. I.n 1954 the unemployment rate for 16- to 24-year-olds 
was 10.6 percent while the overall rate was 5.6 percent. 
Nearly 25 years later, at the end of 1978, the rate for 16- 
to 24-year-olds was at 12.2 percent compared to a 6-percent 
overall rate. 

Such youth unemployment statistics, however, do not be- 
gin to illustrate the magnitude of the problem* Teenagers, 
for example, have much higher unemployment rates than older 
youths. High school dropouts, low-income and minority (partic- 
ularly black) youths also have higher unemployment rates than 
other youths. While the official teenage unemployment rate in 
the ghettos is 40 percent, the Urban League suggests that the 
real rate i.s 60 percent. Also, as pointed out in congressional 
testimony before the House Committee on Education and Labor, 
unemployment statistics do not convey the reduced sense of 
self-worth or the dashed hopes and dreams caused by unemploy- 
ment. They do not convey the pervasive alienation that can 
occur among unemployed youths who feel they cannot make a 
useful contribution to society. 

The seriousness of the youth unemployment problem is a 
matter of controversy. One contention is that it is not 
serious because youths have few financial responsibilities 
and, therefore, are not greatly hurt when unemployed. Another 
argument is that many unemployed youths are students looking 
for part-time jobs, which causes unemployment statistics to 
overstate the true extent of the problem. On the other hand, 
some contend that youth unemployment is a serious problem 
which translates into lower earnings later in life due to 
lost job experience. Also, youth unemployment may result 
in increased crime and alienation which may have long-term 
effects . 

CONGRESSIONAL CONCERN ABOUT YOUTH 
UNEMPLOYMENT LEADS TO NEW INITIATIVES 

In the early 1960s, the Congress determined that youth 
unemployment was a serious problem needing attention. As a 
first step, the Congress provided funds for youth development 



ERIC 



and training under the Manpower Development and Training Act 
of 1962. In 1964, the Economic Opportunity Act established 
two major programs for youths — the Neighborhood Youth Corps 
and the Job Corps. With the passage of the Comprehensive 
Employment and Training Act (CETA) in 1973, youths continued 
to participate in the Job Corps and an offshoot of the Neigh- 
borhood Youth Corps, which was known as the Summer Program 
for Economically Disadvantaged Youth. In addition, youths 
participated to a significant extent in other CETA program 
activities, such as work experience which generally operates 
on a year-round basis. 

Based on a concern about continued high unemployment rates 
for youths, the President in early 1977 requested funds for new 
youth employment programs as part of the job creation component 
of his economic recovery package. The resulting legislation, 
the Youth Employment and Demonstration Projects Act of 1977 
(YEDPA), which amended CETA, authorized four additional youth 
programs . 

YEDPA was an admission that, despite years of experience 
and the expenditure of billions of dollars for youth employment 
and training programs, no one knew how to solve the chronic 
youth unemployment problem. YEDPA is an attempt to test var- 
ious approaches to determine what works best so that effective 
youth programs can be developed. The following prog'-ams are 
YEDPA components : 

1. The Young Adult Conservation Corps — operated jointly 
by the Departments of Labor, Agriculture, and 
Interior — provides employment for out-of-school 
youths 16 through 23 years of age on conservation 
projects on public lands. 

2 . The Youth Community Conservation and Improvement 
Projects (YCCIP) — designed to employ 16- to 19-year- 
old out-of-school youths in well-supervised projects 
with tangible outputs benefiting the community. 

3. The Youth Employment and Training Program (YETP) — 
designed to provide a full range of activities and 
services, including job experience, training, occu- 
pational information, and supportive services for 
in-school and out-of-school 16- to 21-year-olds. 
The program also serves 14- and 15-year-old 



in-school youths. YEDPA authorizes some YETP funds 
to be used for innovative and experimental projects. 
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4. The Youth Incentive Entitlement Pilot Projects 

(Entitlement) — designed tp provide jobs and test the 
efficacy and impact of guaranteeing jobs for all 
16- to 19-year-old disadvantaged youths residing 
in a designated geographic area and attending school 
or who are willing to return to school- 

More than $3 billion has been provided for YEDPA programs 
for fiscal years 1978-80. Most of these funds have been dis- 
tributed by formula to all 460 CETA prime sponsors, usually 
State and local governments, to operate YETP and YCCIP. The 
Congress has provided $222 million for the Entitlement experi- 
ment which is limited to 17 prime sponsors. For these prime 
sponsors. Entitlement funds represent the vast majority of 
their financial resources for youth programs. 

Entitlement, one of the largest demonstration programs 
the Federal Government has ever attempted in the field of 
youth employment, is unique in several respects. Any youth 
who meets the eligibility requirements is guaranteed a Dob. 
It is the Nation's first job guarantee experiment. 

Entitlement is also unique because it requires a commit- 
ment to an educational program as a condition of employment. 
Program participants must remain in or return to school and 
meet established job and academic performance standards. 
Youths work part-time during the school year and full-time 
during the summer. To successfully complete Entitlement, a 
youth should obtain either a high-school diploma or its equiv- 
alent. Youths who perform unsatisfactorily in school or at 
work should be terminated from Entitlement. Youths should 
also be terminated when they become 20 years old or are no 
longer eligible because of family income or the location of 
their residence. 

A third unique aspect of Entitlement is that it gives 
wage subsidies to private employers that provide jobs for 
youths. The legislation allowed Labor to experiment with 
a variety of wage subsidies up to 100 percent to encourage 
private employers to participate. In addition to being a 
demonstration, it is a large-scale operating program- 

Entitlement is typical in that many job experiences are 
identical to those in other CETA programs. Other than 
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private sector worksites (which account for roughly 20 percent 
of participant work hours)/ work locations and the type of 
work performed by program participants were often indistin- 
guishable from other CETA youth program worksites. 

We reviewed Entitlement because it is the most radical 
departure from traditional youth employment programs. En- 
titlement started in March 1978 and v;as originally scheduled 
to terminate in June 1979. The program was later extended 
through September 1980. The President has submitted a youth 
legislative proposal to the Congress which would incorporate 
the concepts of YETP, YCCIP, and Entitlement in a new youth 
initiative . 

Recent GAO reports on youth employment and training 
programs are listed in appendix I . 

PROGRAM RESPONSIBILITY 

The Department of Labor's Office of Youth Programs within 
the Employment and Training Administration has overall responsi- 
bility for supervising, managing; and directing Entitlement. 
However, due to Labor's strained staff capacity, the desire 
to limit Federal workforce expansion. Entitlement's short-term 
nature, and the need for special expertise in evaluation, re- 
search, and demonstration. Labor awarded a contract of about 
$15 million to the Manpower Demonstration Research Corporation 
(MDRC) of New York City to manage and evaluate Entitlement. 

MDRC is a nonprofit, publicly supported, private organ- 
ization which designer, manages, and evaluates social programs. 
It was established in 1974 with support from public and pri- 
vate agencies interested in pursuing innovative ways to treat 
some of society's most difficult and longstanding problems. 
MDRC is governed by a board of directors and has a staff of 
more than 100 persons. 

Under Labor's overall direction, MDRC ' s Entitlement re- 
sponsibilities include (1) helping Labor negotiate work plans 
and budgets for selected prime sponsors, (2) developing and 
supervising demonstration procedures, (3) implementing and 
supervising the research design, (4) implementing an Entitle- 
ment information system, (5) analyzing the consistency of 
operations with demonstration and research requirements, and 
(6) identifying training and technical assistance needs. In 
addition, MDRC helped select Entitlement prime sponsors. 
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COMPETITIVE SELECTION PROCESS USED 
TO SELECT BEST PRIME SPONSORS 

Because of the expected high cost of a guaranteed job 
program. Labor limited Entitlement to a few prime sponsors. 
Labor developed a national competitive selection process 
consisting of a preapplication and a final application phase 
to guarantee the selection of prime sponsors best able to 
meet administrative and research needs. This two-step process 
included site visits before final selection so that the deci- 
sion would be based on more than grantsmanship capabilities. 
The competitive process was designed to yield a wide economic 
and geographic range of sponsors, including large and small 
cities, high-density urban areas, and sparsely populated rural 
regions, and areas of varying unemployment rates, school drop- 
out rates, and racial and ethnic composition - 

Due to cost. Labor limited Entitlement to a few large- 
scale tests and additional smaller tests. Since YEDPA sug- 
gested several secondary experiments and innovations. Labor- 
designed a two-tier approach. Large-scale, or Tier I, projects 
would test the feasibility of implementing a full-scale job 
guarantee program. Each Tier I sponsor was expected to provide 
between 3,500 and 10,000 jobs. Because the Tier I demonstra- 
tions were expected to be burdened by administrative demands, 
Labor allowed smaller. Tier II, projects to test various in- 
novative approaches. Tier II demonstrations were expected to 
provide between 200 and 1,000 jobs. 

All CETA prime sponsors were allowed to submit a preap- 
plication for planning funds to develop a final application. 
Labor requested that each applicant provide specific informa- 
tion concerning: demographic characteristics of the prime 
sponsor area and the proposed Entitlement area, secondary 
school systems and CETA youth programs, proposals for organ- 
izational and administrative arrangements, and a project budget 
and estimates of job demand among eligible youths. 

Criteria used to assess preapplications included the 

— overall quality and thoroughness of the preapplication; 

— managerial, administrative, operating, and fiscal capa- 
bility of the prime !?ponsor; 

level of commitment of other resources to the Entitle- 
ment project; 
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— commitment of cooperation and participation from local 
school systems, labor organizations, and other local 
groups ; 

— degree to which Entitlement would be integrated with 
local education, career development, and employment 
and training programs; 

— previous experience in administering related programs ; 
— proposed cost structure; and 

— commitment and ability to provide data necessary for 
research and analysis. 



Labor established an Entitlement selection and review 
panel composed of 10 individuals from academic, labor, 
community-based organization (CBO), youth population, and man- 
power and social program backgrounds. All panel members ex- 
cept one, a college student, serve on the MDRC board of direc- 
tors . 

Labor received 153 preapplications , 51 for Tier I and 
102 for Tier II projects. Based on the above criteria. Labor 
assessments, and the geographical and socioeconomic distribu- 
tion mandated by YEDPA, the panel made unanimous recommenda- 
tions to Labor. Labor awarded planning grants to support de- 
velopment of final applications to 18 and 16 prime sponsors 
for Tier I and Tier II projects, respectively. 

Labor and MDRC review teams visited each prime sponsor 
while it prepared the final application. The visits were 
designed, among other things, to substantiate preapplication 
information, to explain program requirements, and to assess 
the capabilities and commitment of key operating agencies. 

Labor ranked each final application according to four 
criteria- (1) program management capability, feasibility, 
and commitment, (2) operational feasibility, (3) management 
information and fiscal systems, and (4) budget analysis and 
local resource commitments. A fifth criterion, innovative 
features, was used to rate Tier II proposals. The review 
panel considered all available materials (preapplications, 
final applications. Labor assessments, field visit reports, 
and ratings) and made its recommendations to Labor, which 
selected 17 prime sponsors to operate the Entitlement demon- 
stration. Labor awarded grants to 7 Tier I and 10 Tier II 
prime sponsors. (See app. II.) 
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OBJECTIVES , SCOPE, AND METHODOLOGY 



We examined the following eight areas: 
— Administration . 
— Manage.nent costs • 

— Recruitment, enrollment, and assignment to jobs. 

— Maintenance of effort and substitution . 

— Cooperation and coordination with CBOs , local educa- 
tion agencies , the private sector , and other agencies 
serving youths . 

— Work experience. 

— Academic and attendance standards . 

— Monitoring and evaluation. 

Our review generally covered fiscal years 1978, 1979, and part 
of 1980. 

We made our review at Labor headquarters in Washington, 
D.C., and in 7 of Labor's 10 regions: Atlanta, Boston, Chicago, 
Dallas, Denver, Philadelphia, and Seattle. We visited the head- 
quarters of MDRC in New York City and Abt Associates, Inc., an 
MDRC subcontractor in Cambridge, Massachusetts. 

We visited the 7 Tier I prime sponsors and 4 of the 10 
Tier II sponsors: Albuquerque, New Mexico; Hillsborough County, 
New Hampshire; 1/ Dayton, Ohio; and Philadelphia, Pennsylvania. 
We also visited selected prime sponsor grantees and contractors, 
including local education agencies and CBOs • 

Using standardized questionnaires, we interviewed 848 par- 
ticipants and 460 supervisors at 350 Tier I worksites. We also 
visited 37 worksites at the four Tier II locations and inter- 
viewed 76 participants and 59 supervisors. In most cases. 



l/Entitlement project location was Nashua, New Hampshire. 
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these visits were unannounced. Interviews at Tier I and II 
locations took place in two phases, a surruner phase when the 
participants were working full time and a school year phase 
when the participants were working part time. Summer inter- 
views took place during July and August 1979 r school year in- 
terviews, between September and December 1979. 

Sampling methodology 

We used statistical sampling techniques to evaluate 
youths' Entitlement program work experiences. On the basis 
of a statistical sample of worksites at each Tier I location, 
we obtained information from participants as well as their 
supervisors and evaluated the worksites. This approach al- 
lowed us to draw conclusions about Tier I Entitlement loca- 
tions as a whole. Because Tier II sites were not statis- 
tically selected, the results at these sites are not project- 
able. Tier I sites accounted for 90 percent of the program 
participants through November 1979 and 92 percent of reported 
Entitlement expenditures. 

Statistical sampling enabled us to draw conclusions 
about the universe of interest (worksites) on the basis of 
information on a sample of that universe. The results from 
a statistical sample are always subject to some uncertainty 
(i.e., sampling error) because only a portion of the universe 
has been selected for analysis. Our sample size was large 
enough to keep the sampling error small. The sampling error 
consists of two parts: confidence level and range. The con- 
fidence level indicates the degree of confidence that can be 
placed in estimates derived from the sample. The range is 
the upper and lower limits atween which the actual universe 
value may be found . 

For example, a random sample of worksites showed that 
33 percent were in the private, for-profit sector. Using the 
sampling error formula, we are 95-percent confident that the 
true percentage of private worksites in the universe would be 
within plus or minus 5 percentage points of the sample re- 
sults. Thus, if all worksites in the universe were checked, 
the chances would be 95 in 100 that the actual percentage of 
private, for-profit worksites would be between 28 and 38 per- 
cent . 

Using a worksite as our sampling unit, we devised a two- 
stage stratified sampling plan. From about 4,700 Tier I 
worksites in existence in June 1979, we selected 350 for 
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evaluation — 50 at each Tier I location. This sample size was 
determined so that the expected sampling error on a program- 
wide finding would not exceed plus or minus 5 percentage 
points at the 95-percent confidence level. The sampling 
error on a finding at any particular Tier I location was ex- 
pected to be about 8 percentage points at the 95-percent 
confidence level . 

To select the sites, we stratified all worksites at 
each Tier I location by the number of Entitlement partici- 
pants assigned to the site. Based on this stratification, 
we randomly selected 50 sites at each Tier I location using 
random number tables. At each site selected, we interviewed 
all participants if five or fewer were present. At worksites 
with more than five participants, we used random numbers to 
select five from those present. In addition, we interviewed 
the supervisors of all selected participants. To solicit 
responses from both participants and supervisors, we used a 
structured interview form which was uniform for all locations. 
We used the supervisors' and participants' responses, our 
observations, and a review of records as our basis for asses- 
sing each sampled worksite. 

We then projected the results of these worksite assess- 
ments to what we estimate to be 5,200 worksite locations. 
This worksite estimate rose from 4,700 due to the fact that 
some worksites had subsii.es. We computed these projections 
using weights based on fl) the number of sites sampled in 
each stratum at each location and (2) the number of sites in 
each stratum at each location. The estimates derived from 
the sample of participants and projected to the universe were 
calculated using weights based on the number of (1) partici- 
pants interviewed at each site, (2) participants present at 
each site, (3) sites sampled in each stratum at each location, 
and (4) sites in each stratum at each location. 

We estimate that the participant responses can be pro- 
jected to a universe of about 14,800 Tier I participants. 
The actual number of participants assigned is probably more 
since our projections are based on participants present on 
the days of our site visits. In fact, we estimate that 
there were about 20,000 participants assigned to the 5,200 
sites . 
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CHAPTER 2 



MOST WORKSITES PROVIDED YOUTHS WITH ADEQUATE 
WORK EXPERIENCES WHILE OTHERS NEED IMPROVEMENT 



The Congress specifically mandated ttiat YEDPA programs, 
including Entitlement, offer participants meaningful opportu- 
nities to earn and learn. To carry out this mandate, Labor, 
in its Planning Charter for YEDPA, emphasized the goal of a 
quality youth work experience. 

Based on worksite visits, we estimate that of all Tier I 
worksites in the program 

— 81 percent provided environments where youths were 

encouraged to develop and demonstrate good work habits, 

— 73 percent provided youths with enough useful work, 
and 

— 74 percent provided youths with adequate supervision- 

Although not statistically projectable, the work experience 
characteristics of the Tier II sites we visited were about the 
same as the Tier I sites. 

Most worksites provided all three elements crucial to an 
adequate work experience; however, many others did not. We 
estimate that about 3,000 (58 percent) of the 5,200 worksites 
met our criteria for all three elements, while the remaining 
2,200 (42 percent) lacked one or more of these elements: 
1,100 (21 percent) had serious problems — failing to meet our 
criteria for two or more elements — and another 1,100 (21 per- 
cent) had moderate problems — failing to meet our criteria for 
one element. We estimate that 50 percent of all Tier I par- 
ticipants were assigned to those worksites (42 percent) which 
had one or more problems . 

To evaluate worksite adequacy in any of the three cate- 
gories, we considered supervisor and participant responses 
and rur observations of activities at the site. We also 
analyzed data from our site visits to identify factors that 
were characteristic of adequate and inadequate worksites. 

Although most worksites provided adequate work experi- 
ences, others need improvement. In varying degrees, sponsors 
did not: 
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Assure that youths received adequate supervision. 

— Develop and sustain a sufficient quantity of jobs in 
the right locations. 

— Adequately counsel youths. 

— Provide effective monitoring of worksites. Problems o 
poor supervision, marginal work experiences, and time 
and attendance or payroll violations that should have 
been obvious often went undetected or, if detected, 
uncorrected . 



RESULTS OF WORKSITE ASSESSMENTS 

Labor's Entitlement regulations stress that worksites 
were to have capable supervision and provide structured, pro- 
ductive work settings. The jobs were to introduce youths to 
the habits of successful work life and entry level or prep- 
aratory skills. Additional requirements included develop- 
ing enough jobs to minimize the time between enrollment and 
assignment to a worksite and providing monitorable attendance 
and productivity standards . 



Work habits 

Good work habit development is a critical element in any 
youth employment program. For many participants an Entitle- 
ment job is their first work experience. Based on our inter- 
views with oarticipants, we estimate that 81 percent of the 
Entitlement^ participants never had a prior job. Consequently, 
it is extremely important that these youths be exposed to 
and guided in the development of good work habits. 

On the basis of worksite interviews with participants and 
their supervisors as well as our recorded observations, we 
evaluat-ed whether the worksite provided an environment where 
good work habits were encouraged. In making our evaluation, 
we considered factors, such as: 

— Whether youths came to work regularly. 
— Whether youths came to work on time. 

— How adequately work and progress of participants was 
evaluated . 
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— How adequately services, other than job training, 
were given to participants . 



We estimate that 81 percent of all worksites provided environ- 
ments where good work habits were encouraged. Seventy-three 
percent of the program participants were at these sites. 

We also examined our data to identify factors that were 
characteristic of sites that provided good work habit develop- 
ment. According to our analysis, the adequacy of supervision 
and the quantity of work were the two key factors. We deter- 
mined that youths at sites with adequate supervision and a 
sufficient quantity of work are more likely to exhibit good 
work habits. Conversely, youths at sites with inadequate 
supervision or not enough work to keep them busy are not 
likely to exhibit good work habits. Other factors were con*- 
sidered but did not prove to be important in distinguishing 
between sites that provided or did not provide good work habit 
development. These factors included the number of youths 
assigned to the worksite, the number of youths present on 
the day of our visit, whether the site provided full-time 
jobs in the summer or part-time jobs during the school year, 
and whether the site was private, for-profit or not. 

As discussed above most Entitlement worksites provided 
environments where youths were exposed to real work situa- 
tions and encouraged to develop good work habits. Some 
examples follow: 

— A Federal agency in Cincinnati employed three youths 
as general office workers whose duties included filing, 
typing, and running office machines • The supervisor 
encourages good work habits by treating the partici- 
pants as regular employees . The participants are 
supervised at all times, have enough work, and receive 
training in general office procedures. 

— A supervisor at a Boston shoe manufacturer provides 
constant direction to two participants in the account- 
ing deparument. The participants perform the same 
activities as regular employees and are assigned 
additional work as they complete their tasks. 

— A supervisor in a Cincinnati retail store encourages 
good work habits by assuring that the one participant 
is gainfully employed and adheres to work-hour 
standards. The participant's main duties include 
stocking shelves, unpacking new stock, and cleaning 
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the sales area. In addition, the supervisor is pro- 
viding the participant with sales training • 



Other sites, however, did little to encourage good work 
habits. For example: 

— A large Detroit high school employed 23 students in 
custodial work. The supervisor routinely allowed 
students to leave after they finished 2 hours of work, 
although they were paid for 4 hours. The site did 
not have enough work for the youths assigned. 

— At a Boston youth activities center, four participants 
did no work during the 3 hours we were there. One 
participant read a magazine. Another was sent out to 
get lunch for the other employees. In addition, two 
participants who reported over an hour late for work 
were signed in by their supervisor at their normal 
starting times and paid for the hours not worked. 

' — In Seattle, one youth was assigned to an Armed Forces 
recruiting office as a receptionist and clerk typist. 
In violation of the rules, the supervisor allowed the 
youth to call in sick on several occasions and au- 
thorized payment for these absences- The supervisor 
permitted the youth to use a telephone answering 
machine , so she could go shopping and take extended 
lunch hours. After our visit, the sponsor removed 
the youth from the site and will not use the site 
until a quality supervisor is assigned there. 

Work content 

In evaluating worksites for the usef ulness/meaninrjlul- 
ness of jobs provided youths, we assessed whether tha jobs: 
(1) were useful to the community or employer and (2) provided 
youths an opportunity to learn a new skill or improve exist- 
ing skills. We classified worksites as providing youths with 
useful/meaningful work if at least 60 percent of youths' daily 
activities at the sites met either of these two criteria. 

Overall, we estimate that about 73 percent of the work- 
sites provided youths with useful/meaningful work. Further, 
we estimate that 60 percent of all Tier I participants were 
at these sites* Worksites were more successful in providing 
youths with useful jobs than they were in providing youths 
with jobs where they could develop new or use existing 
skills as shown below: 
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Estimated 
percentage of 
worksites 



Percentage 
of youths 
at sites 



Useful to employer 

or community 
Useful to participant 



74 
66 



62 
56 



We also examined our data to identify factors that were 
characteristic of sites that provided a useful/meaningful work 
experience. According to our analysis, there were four key 
factors., of which quantity of work was most important. 
Supervision, minimizing idle time, and the number of youths 
assigned to a site were also important. Other factors con- 
sidered which did not prove to be important included the 
number of youths present at the worksite, whether the site 
provided full-time jobs in the summer or part-time jobs 
during the school year, and whether the site was private, 
for-profit or not. 

Our analysis indicates that a useful/meaningful work 
experience is not likely to exist at worksites where too many 
participants are assigned, where there is an insufficient 
quantity of work, and where supervision is inadequate. 
Following are some representative examples of worksites we 
evaluated as adequate. 

A Detroit hospital gave 29 students an excellent job 



experience and an opportunity to develop occupational 
skills and good work habits. Twenty-six of the stu- 
dents worked as nurses aides in various departments 
and three performed clerical duties. The site super- 
visor emphasized good work habits, enforced work- 
hour rules, and motivated the youths to accept respon- 
sibility. In return, the youths were providing a 
valuable service. 

■Again in Detroit, a cultural center used six partici- 
pants as program aides. They assisted in planning 
activities, aided students in various workshops, and 
gave tours of the site. The participants were actively 
involved in the operations of the site and had daily 
contact with the public and professionals in the 
areas of art, music, and crafts. In addition to pro- 
vidina a useful service, some of the participants were 
interested in continuing to work in these areas and 
were receiving good training . 
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other sites did not provide a sufficient quantity of 
useful/meaningful work. For example: 

— In Baltimore, five recreation sites employed 45 youths 
as recreation aides. Idleness was a problem at these 
sites, and the youths, when not obviously idle, parti- 
cipated in sports activities or watched other youths 
engaged in recreational activities. The lack of 
enough work and adequate supervision prevented youths 
at most recreation sites from even receiving training 
in rudimentary work habits. 

— The City of Detroit transportation department employed 
22 youths to fill requisitions and take inventory. 
Some youths worked during the day when they were super- 
vised by regular employees and were given a sufficient 
quantity of work to do. Several youths, however, 
worked in the evening when there was an insufficient 
quantity of work and not enough supervision. These 
youths sat around, and some left work early — with pay. 
The supervisor believed he could only supervise and 
keep busy a maximum of 12 participants. 

In substance, most Entitlement jobs were similar to 
those in other youth programs, with clerical, custodial, and 
recreational jobs predominating ^ They tend not to be skill 
oriented in the sense that classroom or on-the-job training 
are prerequisites for performing the job tasks. We estimate, 
however, that about 50 percent of the participants received 
on-the-job training, even though few youths entered the pro- 
gram with training as an objective. In contrast, only about 
6 percent received any formal classroom training . 

THE QUALITY OF YOUTHS' JOB 
EXPERIENCES CAN BE IMPROVED 

Although the majority of Entitlement worksites provided 
an adequate work experience, improvements in several program 
elements are needed to enhance the quality of the youths' job 
experience. supervision, job development, counseling, and 
worksite monitoring are all important elements in providing 
quality job experiences, with supervision the most important 
element . 
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Supervision of youths 



In analyzing the factors contributing to the quality of 
the work experience, supervision was a significant factor 
in work habit development and meaningful work content. 

In evaluating supervision, we assessed how adequately 
supervisors 

— understood program objectives; 



— enforced work— hour standards; 
— controlled participant behavior; 
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--minimized idle time; 

— impressed upon participants the need for good work 
habits ; 

— assigned, trained/ and/or guided participants in their 
job tasks; and 

— evaluated participant work and progress. 

We obtained information on these issues by interviewing 
worksite supervisors and participants, observing worksite 
conditions, and examining records. 

We estimate that 74 percent of the worksites provided 
youths with adequate supervision. These sites had 67 per- 
cent of all Tier I participants. 

We also examined our data to identify factors that were 
characteristic of sites that had adequate supervision. We 
analyzed the following factors: 

— Number of youths assigned to the worksite. 

— Number of youths present on the day of our visit. 

— VThether enough work was available. 

— Whether the site provided full-time jobs in the 
summer or part-time jobs during the school year. 

— Whether the site was private, for-profit or not. 

2.9 
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— Whether the site was in an urban or rural area. 
— Whether attendance was recorded. 

— Whether attendance was taken in a timely manner. 

— Whether attendance records were complete. 

According to our analysis, only three of these factors 
were significant in separating sites with adequate supervision 
from those without. In order of importance, these factors 
were: whether enough work was available to keep the partici- 
pants busy, whether attendance records were complete, and 
whether attendance was recorded at all. Once the effect of 
these factors was considered, none of the others was signifi- 
cant. Examples of sites exhibiting significant characteris- 
tics of adequate supervision included the following: 

— Six participants at a Boston high school worked on a 
general maintenance crew. They worked side-by-side 
with the regular staff preparing the school for 
occupancy in the fall . Their job tasks included 
washing the windows and floors, landscaping, and 
painting. The supervisor had a schedule of projects 
to be completed before school began. The supervisor 
enforced work -hour standards, minimized idle time, 
and encouraged good work habit development. 

At a Nashua recreation facility, four participants 

were engaged in different jobs. Two youths assisted 
their supervisor in preparing and serving food. The 
supervisor scheduled their activities allowing for 
little idle time. Another participant worked in a 
clerical position answering the phone, typing, pre- 
paring the payroll, and enrolling new club members. 
The fourth participant was a recreation aide and 
distributed equipment and monitored various youth 
activities . 

— At a Cincinnati special education center, one partici- 
pant was learning to type, operate office machines, 
and prepare educational material . The superviscpr 
maintained constant supervision over the participant 
and assured that she was kept busy. 
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Where too many participants were assigned for the amount 
of work available and work-hour standards were not enforced, 
supervision was generally inadequate. For example: 

A Mississippi high school worksite had over 100 par- 
ticipants assigned primarily to custodial functions. 
Some worked as teachers aides. The Vice Principal 
readily admitted that the crew was too large; however , 
he tolerated the situation because the participants 
needed the money. We observed a great deal of milling 
around, little useful work, and no intervention on the 
part of the supervisors to at least minimize the chao- 
tic conditions at this site. For example, we observed 
one youth vacuuming a few square feet of floor for 
about 30 minutes while talking to his friends. 

A large Cincinnati worksite employing over 100 parti- 
cipants divided the youths into groups to clean vacant 
lots throughout the city. The supervisor of one group 
told us that participants are frequently absent, and 
she never knows how many youths will appear each day. 
Further, no action was taken to discipline youths who 
do not work or to encourage better attendance. 

Fifty-two participants at a Baltimore recreation center 

were doing no work. They either were playing games 
or were simply idle. Five participants took a trxp 
to an out-of-state amusement park. One youth skipped 
a day's work because he was to attend a professional 
baseball game the next night as a paid activity. 
According to the worksite agreement, the youths were 
supposed to be working in maintenance, clerical, and 
recreational positions. Further, youths were paid for 
lunch periods and there were numerous time and attend- 
ance discrepancies. These activities occurred with 
the supervisor's approval and no attempt was made to 
put the youths to work. One supervisor was fired and 
three others were suspended after we brought this 
situation to the attention of program officials and 
they had confirmed our findings. 

We estimate that supervisors did not properly prepare 
or complete attendance records at 32 percent of the worksites. 
We also estimate that at least one youth was absent at about 
22 percent of the worksites on the day of our visits, and at 
26 percent of these sites at least one youth was paid for the 
absence. This tended to be a problem at the larger sites. 
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In some cases, improper payments were caused by inade- 
quate time and attendance and payroll controls. In other 
cases, supervisors allowed participants to presign timesheets 
or fill them in at the end of a pay period instead of at the 
end of each day. Others authorized payment because they 
viewed Entitlement as simply an income transfer program. 
Some examples of time and attendance problems follow: 

— In Boston, we reviewed the timecards of the 17 parti- 
cipants absent on the day of our visit. A sponsor 
official told us that absenteeism was a problem and 
estimated that, on any given day, 20 percent of all 
participants failed to report to work. Of more sig- 
nificance, however, was the fact that 7 of the 
17 participants (41 percent) were paid for the day 
they were absent. 

— Baltimore had similar problems. At seven sites we 
visited, 15 (47 percent) of 32 youths absent on the 
day of our visit were paid. Baltimore worksites 
also had significant time and attendance record- 
keeping problems. (See p. 18.) 

The worksite supervisor clearly has prime responsibility 
for enforcing time and attendance requirements. V7e believe 
that improvements in this area would lead directly to better 
participant work habits. 

Job development 

Tc fulfill the Entitlement job guarantee. Labor regula- 
tions required sponsors to develop jobs which were easily 
accessible to participants and reasonably close to their 
homes. Further, there had to be enough jobs so that the 
waiting time between enrollment and job assignment was 
minimized. Generally, sponsors relied on the traditional 
providers of jobs from the summer youth program; e.g., city 
and State agencies, CBOs, and schools. Initially, they 
appeared to have developed a sufficient quantity of jobs. 
However, most Tier I sponsors had difficulty creating or 
maintaining an adequate number of part-time jobs during the 
school year. In addition to problems in developing jobs, 
some sponsors had administrative problems in the timely 
assignment of youths to available jobs. 
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Mississippi found it particularly difficult to maintain 
enough jobs. Ironically, it had done a fairly good job in 
assigning youths at the start of the program. However, when 
its program shifted from full-time summer jobs to part-time 
school year jobs in both 1978 and 1979, a severe deficit in 
the quantity of accessible jobs occurred. As late as Decem- 
ber 1979, about 18 percent, of the enrollees were in a hold 
status pending assignment or reassignment to a job. The 
lack of transportation, school/work scheduling problems, and 
coordination difficulties, mainly between the sponsor and 
its major subcontractor, have contributed to the problem. 

Other sponsors also had difficulties in developing 
accessible part-time jobs that met Entitlement's 10-hour per 
week minimum work requirement during the school year. The 
location of available jobs, school scheduling problems, and 
transportation difficulties affected the sponsors' ability 
to fulfill Entitlement's job guarantee. 

To resolve some of these problems, Detroit made liberal 
use of fast food establishments because they provide flexible 
hours. School sites were also a major source for placements; 
however, many were overloaded. The sponsor's program manager 
told us that he considers school sites as a last resort 
because they provide low quality work experiences. In 
February 1980, the sponsor placed a moratorium on further 
assignments to them due to the overloading. 

In response to a critical job shortage, Cincinnati cur- 
tailed its active outreach efforts in February 1980. A 
sponsor official stated that by 1980 the public sector was 
saturated, but they were attempting to place additional youths 
at existing public and private worksites. 

At the inception of the program, the many participants 
who claimed their guaranteed job overwhelmed the assignment 
capabilities of most Tier I sponsors. Some participants 
waited months for jobs as sponsors' assignment processes 
faltered. Most Tier II sponsors found it easier to assign 
participants to jobs because of their smaller numbers of par- 
ticipants. As the program progressed, most sponsors reduced 
the time between enrollment and assignment. In a March 1980 
report, MDRC attributed much of the improvement in assignment 
processing times to better administrative procedures, partic- 
ularly in Boston and Detroit. Of some significance, however, 
is the fact that sponsors had far fewer youths entering the 
program than they did at the beginning of the program. 
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Further, many youths were on hold and not working pending 
initial assignment, reassignment, or termination. MDRC re- 
ported that as of December 1979, about 2,200 youths were on 
hold pending a first assignment, 59 percent of whom had waited 
more than 3 0 days. Another 1,800 youths were on hold pending 
reassignment or termination* Eighty-seven percent of these 
youths had been in this status longer than 30 days. 

Counseling 

According to their grant applications, all Tier I spon- 
sors planned to provide varying degrees of counseling to 
participants on such matters as education, health, career 
or job placement, and work habits. Some sponsors planned to 
furnish these services directly, while others planned to con- 
tract with other agencies for some services. 

Although all participants might not have required a fvill 
range of counseling services, many needed school performance 
and work habit counseling. In general, many Tier I sponsors 
placed a low priority on counseling compared to , other program 
components. However, more youths were terminated from En- 
titlement for negative reasons than positive ones. We believe 
this high percentage of negative Lerminations may indicate a 
need for more substantive counseling efforts. 

Through December 1979, an MDRC breakdown of the 34,000 
terminations shows the following: 



Percent 



Positive : 

High school graduates 26.7 
Negative : 

Dropped out of school 5.8 
Unsatisfactory school 

performance 3 . 9 

Unsatisfactory job performance 18.2 
Lost contact 6 . 2 34.1 

Neutral : 

No longer eligible 10.0 
Left for other program or job 11.7 
Other 17,5 39,2 

100.0 
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At most Tier I locations, "counselors" spent little 
time counseling youths. They were primarily involved in the 
payroll process— picking up timesheets and delivering checks. 
Further, many files of participants we interviewed in the 
summer 1/ lacked adequate documentation on counseling activi- 
ties. About 45 percent of these files had no evidence of any 
counseling activity. Thus, in many cases it is impossible to 
determine whether active or terminated participants were ever 
counseled. 

Of the files containing evidence of counseling, about 
80 percent indicated that participants received some counsel- 
ing on work habits. Only about 40 percent contained documen- 
tation on school performance counseling. In our interviews 
with participants, less than half stated that they had dis- 
cussed their future after graduation with their counselor. 

some sponsors did not develop a plan to help participants 
graduating from high school obtain unsubsidized employment. 
In this regard, Baltimore and Boston, when faced with a large 
number of terminating seniors, maintained the 
in their Entitlement jobs by funding them under another CETA 
program, such as public service employment. 

Participants at some of the Tier II sponsors we visited, 
particularly Albuquerque and Nashua, received more frequent 
and in-depth counseling . We believe that the smaller size of 
these programs aided these efforts. 

Monitoring 

Labor's evaluations of other youth programs showed that 
participants benefit most from structured and demanding work- 
site activities. Labor also found that effective monitoring 
dramatically improved worksite quality, particularly when 
sponsors measured whether participants were actively employed, 
productive, and supervised and met attendance and performance 
standards. Consequently, Entitlement guidelines emphasized 
that sponsors not only provide supervised and disciplined 
work experiences but also monitor worksite quality. 



1/We did not review counseling files in the fall 



5 



ERIC 



22 



Initially, most sponsors did not implement effective 
monitoring systems. Although Entitlement started in the 
spring of 1978, sponsors did not begin to seriously monitor 
worksites until the spring and summer of 1979. Cincinnati 
and Detroit did no worksite monitoring during the first 
year of the program. Cincinnati monitored many of its work- 
sites over a period of about 1 month in 1979 in preparation 
for our review. 

Some sponsors lacked effective-! monitoring instruments 
and used untrained personnel as monitors. This resulted in 
some Tier I sponsors performing little more than limited 
compliance reviews of worksite time and attendance data and 
payroll problems. Baltimore, Boston, Cincinnati, and Miss- 
issippi 1/ used checkoff lists to record supervisor and par- 
ticipant responses, but monitors commented infrequently or 
superficially on the quality of worksites. Baltimore and 
Denver used inexperienced CETA public service employees as 
monitors without giving them any specific training. 

Increased sponsor monitoring during and after the summer 
of 1979 did not necessarily lead to the resolution of some 
serious problems. Problems of inadequate supervision, in- 
sufficient quantity of work, and time and attendance infrac- 
tions that should have been obvious went undetected, or, if 
detected, uncorrected until our review. For example, as 
previously discussed, Baltimore and Boston had numerous 
time and attendance and payroll infractions. Nearly all of 
the 50 sites we visited in Baltimore had one or more of the 
following timekeeping or payroll deficiencies: 

— Paid absences, latenesses, and meal breaks. 

Participants maintaining or possessing their own 

timecards . 

— Timecards signed in advance or at the end of a time 
period instead of daily. 

— Signatures without hours shown or vice-versa. 
— Different signatures for the same name. 



1/After the completion of our fieldwork, Mississippi told us 
~ that, in addition to the checklists used by its program _ 

monitors, an independent monitoring unit prepares narrative 

reports on worksite conditions. 
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— Timecards not showing actual days or hours worked. 
— Incorrect totals on timecards. 
— Unapproved changes on timecards. 

— No sign-in sheet, unsupervised sign-ins, or sign-in 
sheets not showing actual hours worked. 

— Sign-in sheets inconsistent with timecards* 

Boston had many of the same problems. Both sponsors took 
corrective action when we brought these problems to their 
attention, and in late 1979, both revised their monitoring 
procedures. The changes in their monitoring procedures 
occurred in the latter stages of our review; consequently, 
we are unable to comment on their effectiveness. 

In some cases, problems at worksites may have gone un- 
detected simply because of the infrequency of sponsors ' moni- 
toring visits. Because of program management cost ceilings 
discussed in chapter 3, Cincinnati, Denver, and Seattle did 
not adequately staff monitoring units. Of the 50 worksites 
we visited, Detroit monitors had never visited 18 and had 
visited 13 others just once. Monitoring reports are super- 
ficial, and for some of the same sites we visited, the monitors 
either did not detect problems that should have been obvious 
or failed to record them in their reports. In Boston, 26 of 
the 50 worksites we visited failed to meet at least one of 
the three elements crucial to an adequate work experience. 
Sponsor monitors had visited 11 of these 26 sites but identi- 
fied problems at only 4. The sponsor had taken no action 
to correct even the problems raised by its own monitors. 

While we recognize that these sponsors had other prob- 
lems, we believe effective and timely monitoring is essential 
in maintaining quality youth programs. 
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CHAPTER 3 



OPERATIONAL PROBLEMS HAMPERED 

PROGRAM PERFORMANCE 

Sponsors had many serious operational problems in imple- 
menting Entitlement. They often found it difficult to comply 
with Labor regulations and MDRC directives and achieve estab- 
lished program goals. Some problems were caused by the rapid 
program startup' and were resolved early. Other problems were 
more persistent, and some new problems surfaced during the 
second year. 

The operational problems were so severe at some locations 
that it is difficult to tell how the program would have been 
implemented had effective management been employed. Some of 
the more difficult problems faced by sponsors in implementing 
Entitlement were: 

— Entitlement was difficult to implement because of its 
dual role as a demonstration and an operating program. 
As the program progressed, administrative systems often 
failed to meet program needs. These problems and high 
staff turnover led to major reorganizations and program 
instability at most Tier I locations. 

— Program management costs were higher than originally 
planned and averaged about 37 percent of total costs. 
The major cause was underenrollment of youths coupled 
with sponsor reluctance to reduce administrative 
staffs . 

--Sponsors had difficulty in enforcing the academic 
and school attendance requirements of Entitlement. 
In some cases, the standards were so 1' beral as to be 
of limited value. In other cases, spo sors did not 
establish the necessary processes to enforce them 
consistently . 

— Sponsors had mixed results in establishing links 
and improving coordination with other groups which 
could assist them in better serving youths. They 
often found it difficult to establish and/or maintain 
cooperative relationships with schools, CBOs, and 
other agencies serving youths. 
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— Entitlement is the first youth employment program which 
makes extensive use of subsidized private sector 
jobs. However, almost half of the jobs are the type 
that youths are normally hired for. The problem of 
potential substitution is compounded by the fact that 
most sponsors initially provided 100-percent subsidies 
to all employers, including those in the private sec- 
tor. Sponsors did little to assure that the Entitle- 
ment participants were not simply a substitute for 
youths who would have been hired without a subsidy. 

ADMINISTRATIVE PROBLEMS AND STAFF 
TURNOVER' LED TO MANY REORGANIZATIONS 
AND PROGRAM INSTABILITY 

Entitlement, because of its dual role as a demonstra- 
tion and an operational program, was difficult to implement. 
Initially, many sponsors could not cope with these multiple 
responsibilities. Delays in job assignments and problems in 
payroll and management information systems were caused by 
the large influx of applicants. 

As the demonstration progressed, administrative systems 
failed to meet program needs. For some sponsors, program 
suspensions and major reorganizations were required. Staff 
turnover became a problem. Although some problems were 
resolved, some continued into 1980. 

Baltimore used the Summer Youth Employment Program as a 
management model for Entitlement. Due to serious management 
control problems, however, this model proved to be inadequate 
for Entitlement. As a result, Baltimore reorganized twice 
during the first year. The sponsor could not cope with the 
paperwork requirements during the summer of 1978. CETA 
public service employees used for administration and worksite 
monitoring were unqualified for the tasks and further compli- 
cated Ihe problem. 

Paperwork logjams prevented tracking participant assign- 
ments. Transfers of job assignments were not documented, 
and the computer rejected timesheets which did not match 
recorded worksites. Thus, hundreds of participants were not 
paid on time. The sponsor suspended program operations on 
two occasions and used the time to correct these problems. 
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In mid-1979, a third reorganization • established two new 
units. The first was designed to monitor worksites and 
troubleshoot operational problems; the second, to monitor 
participants' academic and attendF.nce performance. At the 
time of our fieldwork, Baltimore still did not have an effec- 
tive system for monitoring worksites and school attendance 
and academic performance. 

Boston had numerous problems implementing Entitlement. 
Its large centralized administrative structure proved inade- 
quate in coping with program requirements. The job assign- 
ment system faltered from the program's inception. Hundreds 
of youths failed to receive assignments for months. In April 
1979, the placement unit was reorganized and the backlog of 
job assignments was decreased. However, in early 1980, it 
still took about 6 weeks for a youth to receive a job assign- 
ment after enrollment. Staff turnover disrupted the perform- 
ance of some units. The caseworkers, who are the main link 
between the worksite, the participant, and the program's 
administration, were the most transient. 

Detroit experienced enrollm.ent, payroll, and information 
system problems so severe that the program was suspended for 
2 months. The program was completely reorgaiaized in January 
1979. MDRC officials told us this program has turned around. 
However, as of December 1979, some of the origi;ial problems 
still existed. Our review showed that participcjnts ' files 
contained inaccurate information, paychecks were late or in- 
accurate, and inta]ce centers were not submitting timely status 
changes on participants. 

Cincinnati's high turnover of program managers contrib- 
uted to not implementing a uniform program among contractors. 
At the time of our fieldwork, three individuals har" served 
as program manager. The first did not exert a strong influ- 
ence and spent minimal time with the contractors. The second 
failed to gain needed cooperation, complained of a lack of 
authority, and resigned after 5 months. The third appea^-ed 
to be taking a more active leadership role. 

Denver has been unable to implement Entitlement despite 
repeated intervention by Labor and MDRC. (See pp. 49 and 50.) 
The program has been essentially put on a "life support sys- 
tem" for the duration of the demonstration. In its first 
16 months, the program had four directors and one acting 
director. According to MDRC, with eac?i change Denver's 
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Entitlement structure also changed, so the program never 
stabilized long enough to create confidence or allow any 
participating agency or individual, including MDRC, to 
clearly understand program operations in Denver. 

Although MDRC's March 1980 report concludes that Entitle- 
ment is on the whole feasible, it cautions against rapid im- 
plementation of future programs as demanding as Entitlement. 
The report further states that: 

'* * * [although] substantial improvements in 
administration are noted, * * * the program 
model continues to pose a challenge and the 
effects of the rapid start-up are still 
apparent at some sites." 

We agree that improvements have been made . We question, 
however, whether recurring problems 2 years into the program 
can be attributed to startup difficulties. 



PROGRAM MANAGEMENT COSTS 
ARE~HIGHER THAN EXPECTED 

Most other CETA programs limit the percentage of funds 
which sponsors can use for program adminifitration . For En- 
titlement, on the other hand, MDRC combined program adminis- 
tration with other categories, such as client services and 
worksite supervision, called them program management costs, 1, 
and placed r .it on them. The original grant agreements 

did, ho-vr. .X, contain approved participant /management cost 
ratios. As shown below, program management costs as a per- 
centage of total costs were originally projected to average 
28 percent at Tier I locations. From the program's inception 
through September 30, 1979, management costs had averaged 
37 percent, or about one-third higher than originally 
projected . 



1/Program management costs include all costs except partici- 
pant wages and fringe benefits. 
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E s^t imated an d Actual Program 
Manag e ment Costs as a Percentag e 
of Total Costs 



Sponsor 



Estimated in 
original 
grant 



Percent 
estimated in 
grant 
extension 



Actual through 
September 30, 
1979 



Baltimore 

Boston 

Cincinnati 

Denver 

Detroit 

Seattle 

Mississippi 



28 . 5 
38, 5 

29 .8 
28 . 3 

22 . 3 

30 .9 

23 - 1 



39 .5 
41 .2 
37 .5 



39 .6 
36 .2 
32 .6 



(a) 



35.4 
45 .0 
39 .0 
42 . 1 
37 .9 
42 . 5 
25.6 



Average 



28 . 1 



38 .1 



37 . 2 



a/Denver did not have an approved grant extension during our 
~ fieldwork. 

As shown in tht above table, higher program management 
costs were allowed for grant extensions. In an April 1979 
operations memorandum to Entitlement directors, MDRC was 
obviously concerned that disproportionate ratios might en- 
danger the acceptability of Entitlement once the demonstra- 
tion was over. In this regard MDRC stated: 

"We should all understand that we are not trying 
to penalize anyone, but only trying to maintain a 
demonstration in which the costs and management 
accountability are such that it has a reasonable 
chance of being accepted as an on-going policy 
vehic le • " 

MDRC reported that through August 1979, Tier II sites 
devoted an even higher percentage of funds to management 
costs (41 percent) than Tier I sites (37 percent). 

The variance in program management costs among the spon- 
sors together with a variance in the average number of hours 
youtns actually worked is reflected in the following full 
year cost estimates for participants through September 1979. 
Our estimates are based on the assumption that a participant 
remains in Entitlement for a full year. 
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Sponsor 



Total annualized 
cost per 
participant 



Baltimore 

Boston 

Cincinnati 

Denver 

Detroit 

Mississippi 

Seattle 



a/$3,706 
5,797 
4,621 
5,294 
5, 129 
4,395 
5, 164 



Average 4 , 619 

a/Baltimore's costs are lower principally because it limited 
participants' work to 15 hours during the school year and 
30 hours durin'5 the summer, compared to other sponsors, 
which allowed 20 and 40 hours, respectively. 



A major cause of high program management cost rates was 
underenrollment of participants. Sponsors hired management 
and administrative staffs in anticipation of enrollment level 
that never materialized. MDRC estimates that sponsors were 
successful in enrolling 40 percent of the eligible youths in 
the first year of the program. According to MDRC, most spon- 
sors in their initial applications overestimated the eligible 
population in order to receive enough funds if all eligible 
youths claimed a job. 

In addition to the overestimates of eligible population, 
sponsors may have overestimated the attractiveness of the 
program and underestimated the impact of competition from 
other youth programs which do not require a commitment to 
remain in or return to school. Strong local economies in 
Seattle, Denver, and Nashua which allowed youths to obtain 
unsubsidized employment also contributed to underenrollment. 

All sponsors faced problems in recruiting and retaining 
out-of-school participants. About 10 percent of Entitlement 
participants were not in school when they enrolled in the 
program. Out-of -school youths may iiot be attracted to a 
program paying the minimum wage, with work-hour limitations 
of 20 hours per week or less and requiring a school commit- 
ment. Initially, Boston and Mississippi had insufficient 
alternative education facilities to accommodate those drop- 
outs who enrolled in the program. This deficiency was 



overcome through the establishment of additional alternative 
education programs . 

Retaining former dropouts, even in an alternative school 
setting, has proven difficult. For example, Boston enrolled 
1,200 dropouts. As of December 1979, 500 were attending an 
alternative education program or a regular school. Of the 
700 youths terminated, only 44 (6 percent) received a high 
school diploma or its equivalent. About 90 percent of all 
terminations were due to attendance problems or lack of 
interest. Attendance problems and lack of interest in alter- 
native education programs are not unique to Boston — several 
other sponsors experienced the same problems . 

In addition to enrollment levels, the comprehensiveness 
of services also affects the management cost rates. As dis- 
cussed in other sections of this report, plans to counsel 
youths, monitor worksites, and enforce academic and attend- 
ance standards were not fully implemented. To the extent 
that these services are upgraded or expanded, management 
cost rates may increase. 

DIFFICULTIES IN ASSURING THAT PARTICIPANTS 
MEET ACADEMIC AND ATTENDANCE STANDARDS 

A unique feature of Entitlement is that the job guaran- 
tee is designed to be an inducement for economically dis- 
advantaged youths to stay in or return to school to obtain 
a high school diploma or its equivalent. Therefore, as a 
condition of an Entitlement job, participants must meet the 
minimum academic and attendance requirements of the secondary 
school or high school equivalency program in whxch they are 
enrolled or be terminated from the program. To assure that 
participants attend school, academic and attendance perform- 
ance is to be monitored monthly. 

Initially, some school systems had no school standards 
for sponsors to monitor and enforce- Therefore, Labor re- 
quired that sponsors establish standards for Entitlement 
participants. Some sponsors established standards which 
were more stringent than existing school standards. In some 
cases, students had to pass only one course. In other cases, 
they had to maintain a "C" or higher average. Attendance 
standards also varied considerably. See appendix III for a 
complete list of academic and attendance standards. 
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Enforcement of standards was inconsistent and often 
negligible. In March 1980 MDRC reported that sponsors 
were reluctant to terminate youths and were slow to enforce 
standards. It cited several factors affecting Entitlement 
sponsors' termination policies. 

— In addition to operational problems in establishing 
and enforcing standards, sponsors gave low priority 
to this area because of other systems' problems. 

— Sponsors were inclined to maintain enrollment levels 
to enhance their reputations. 

— Because of their desire to help poorly performing 

youths, sponsor personnel were reluctant to terminate 
them . 

Further, MDRC reported that sponsors were struggling to main- 
tain participant/management cost ratios. Participant termi- 
nations could adversely affect their ratios and result in 
cuts in management funds. 

We reviewed available school records for participants 
interviewed, but included only those who had been in the 
program prior to the summer of 1979. Data on 22 percent 
were either unavailable or unusable. For the remainder, our 
review showed that about 30 percent of active participants 
were not meeting school performance standards. MDRC ' s 
Entitlement information system shows that only 4 percent of 
34,000 program terminees left the program because of un- 
satisfactory school performance through December 1979. 

Sponsors had fairly elaborate systems involving warnings, 
appeals, and hearings for participants not meeting school 
standards. Baltimore and Boston also placed their own staff 
as liaisons in the schools to facilitate the collection of 
school performance data. Although these sponsors terminated 
youths more frequently than other sponsors, their sy lems 
were often ineffective. 

Boston liaisons recommended terminating 466 youths for 
inadequate school performance; however, only 64 were termi- 
nated. These 466 youths had received two warning notices 
and a notification of pending termination. Our sample of 
Boston participants included one participant who, although 
enrolled, did not attend school the entire 1978-79 school^ 
year but worked in an Entitlement-sponsored job. Four other 
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participants missed from 111 to 167 days (62 to 93 percent) 
of the 180-day school year, but none were terminated. These 
absence rates far exceeded the 45 days per academic year 
Boston allowed for unexcused absences. In addition, these 
participants did not meet the minimum academic requirements. 

Although Baltimore had terminated a higher percentage of 
youths for not meeting school standards than other sponsors, 
its system needed improvement. The sponsor's academic com- 
pliance unit was extremely disorganized. Personnel did not 
understand their function or the system. We reviewed the 
files for 164 Baltimore participants and found that many 
lacked the data necessary for evaluating a participant s 
school status. Other sponsors did even less in enforcing 
school performance standards. Detroit and Seattle were the 
lowest of Tier I sponsors with less than 1 percent of their 
terminations for this reason. 

COORDINATION BETWEEN PRIME SPON SORS AND OTHER 
ORGANIZATIONS SERVING YOUTHS NEEDS IMPROVEMENT 

YEDPA required that sponsors establish links with 
schools, CBOs, and other agencies serving youths. All spon- 
sors we visited made some progress in obtaining commitments 
from local institutions and organizations. However, a number 
of important relationships were not always lasting or without 
conflict. At some locations, this inability to work coopera- 
tively caused significant problems. 

Sponsors' relationships with schools 

By design. Entitlement required a special relationship 
between sponsors and schools. Schools provided easy access 
to potential participants, were essential to providing spon- 
sors with academic and attendance data on participants, and 
could assist sponsors' job development efforts by arranging 
flexible school schedules for participants. At some loca- 
tions, schools played an important role in managing the 
program . 

Detroit initially contracted with the school system to 
implement the program. As discussed further S'" P^5^^^°^^^^^^ 
relationship deteriorated to the point where Labor and MDRC 
required a suspension of program operations for - months. 
Later, the sponsor assumed responsibility for managing the 
program . 



In Denver, the sponsor and the i school system discontinued 
their relationship during contract renegotiations. The school 
department terminated the relationship because school offi- 
cials did not believe their funding level was commensurate 
with their responsibilities. This wa3 one issue contributing 
to the Labor and MDRC decision to suspend new enrollments 
in Denver from June 1979 to at least June 1980. (See p. 49.) 

As previously stated, most Tier I sponsors had major 
problems in obtaining complete and timely data on the academic 
performance and attendance of participants. Also, the spon- 
sors were not very successful in securing early releases from 
school for Entitlement participants. This affected job de- 
velopment efforts in Cincinnati , Baltimore, Detroit, Seattle, 
and Mississippi. Some of these sponsors were able to secure 
early releases for participants on a case-by-case basis, but 
generally not on a large scale. 

Sponsors' relationships with 
CBOs and other agencies 

All Tier I sponsors established relationships with CBOs, 
but some important ones were strained to the point of ter- 
mination. In some cases, CBOs were dissatisfied with their 
roles, while in others, sponsors were unhappy with CBOs' 
performance. These problems had significant impacts on im- 
plementing Entitlement at some locations. 

Boston and a major CBO never resolved serious differ- 
ences between them. According to CBO officials, the sponsor 
had led them to believe the CBO would play a major role in 
operating the program. In its grant proposal, the sponsor 
stated that the CBO would participate in intake, job develop- 
ment, and counselor training for the duration of the program. 
In January 1979, the sponsor reduced the CBO ' s responsibili- 
ties to the outreach and intake of out-of-school youths. The 
sponsor and the CBO constantly disagreed and exchanged accu- 
sations during the 19-month contract period. The sponsor- 
claimed the CBO was not meeting its recruiting goals, while ^ 
the CBO claimed that the sponsor took too long getting jobs 
for youths it referred. The situation also deteriorated 
because the CBO failed to fulfill its commitment of providing 
jobs under another program to Entitlement eligible youths 
and graduates for the summers of 1978 and 1979, respectively. 

Mississippi and two of its major contractors, the Uni- 
versity of Southern Mississippi and the State Employment 
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security Commission, had serious problems of cooperation and 
coordination. Under its contract with the sponsor, the uni- 
versity was to provide services, such as transportation, 
alternative education, and the supervision of counselors. 
It was also to coordinate these activities with the school 
districts. In addition, the university was responsible for 
establishing a management information system. The university 
did not perform satisfactorily, and the contract was not re- 
newed when it expired in June 1979. Mississippi then, nego- 
tiated contracts with four CBOs and individual school dis- 
tricts to perform these functions. 

Mississippi and the State Employment Security Commission, 
which was responsible for job development, were often at odds. 
The sponsor stated that the agency was not diligent in devel 
oping jobs. The agency stated that inadequate transportation 
and inflexible school schedules were the real causes for the 
lack of jobs. 

Denver's relationship with two nationally recognized 
CBOs ended in failure. Under their contracts with the spon- 
sor, the CBOs were to recruit 1,100 out-of-school youths and 
enforce alternative education attendance standards. The CBOs 
fell far short of their recruitment goals, and class attendance 
standards were not enforced. Dissatisfied with this perform- 
ance, Denver did not renew their contracts when they expired 
in June 1979. Denver did not have a contract with anyone 
to handle out-of-school youths until February 1980, when it 
negotiated a contract with a State university. 

AC HIEVING A NET INCREASE IN PRIVATE 
SECTOR JOBS FOR YOUTHS IS DIFFICULT 

YEDPA requires that Entitlement jobs not result in sub- 
stituting currently employed workers by program youths. As 
the program progressed, total private sector hours worked 
incrLsed from 14 percent in September 1978 to 23 percent in 
January 1980. While private sector jobs are generally per- 
ceived as better work experiences for youths, there is po- 
tential for substitution because of the subsidy used to 
induce employers to participate. 

The act allowed sponsors to pay various subsidies up 
to 100 percent of the wages for youths employed in the 
private sector, without limiting how long the subsidy was 
to be paid. This provision permitted Labor to test what 
subsidy levels are necessary to encourage private employer 
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participation in the program. With the exception of 
Mississippi at 75 percent, all Tier I sponsors initially 
developed private sector jobs providing for 100-percent 
subsidy . 

Although we did not conduct an indepth review of the 
extent that Entitlement participants displaced currently 
employed workers, we found evidence indicating that Labor 
and the sponsors need to more closely monitor the payment of 
wage subsidies r Of approximately 4,000 private sector jobs 
held by youths during our fieldwork, we estimate that about 
45 percent were with employers who normally employ youths 
for these jobs. This estimate is based on our review of 
sponsor job lists, interviews with supervisors and partici- 
pants, and observations at worksites. The following examples 
indicate the problems with wage subsidies. 

Detroit used many fast food restaurants as worksites. 
Several fast food managers told us that they treat Entitle- 
ment participants as regular employees and use them inter- 
changeably with employees they pay. Although the sponsor 
monitored for substitution, we found evidence of substitution 
at 10 of 26 private sector worksites. During our fieldwork, 
Detroit was expanding the use of fast food restaurants, thus 
increasing the possibility of additional substitution. 

In Boston, we found substitution at two of five private 
employers who normally hire youths. Both employers told us 
that their previous employees were about the same age as the 
Entitlement participants. Although one employer had hired 
an Entitlement participant before the departure of his regular 
youth employee, he replaced the regular employee with another 
Entitlement participant. The other employer not only replaced 
his regular youth employee with an Entitlement participant, 
but hired three other Entitlement participants to do the same 
work that one regular employee did. Both employers stated 
that they would hire unsubsidized employees if the Entitle- 
ment participants were not available* 

In Mississippi, 14 of 20 employers who normally hire 
youths stated they would hire an unsubsidized employee if 
the Entitlement program was not available. Three employers 
had Entitlement eligible youths working for them when they 
became Entitlement job sponsors. The youths enrolled in the 
Entitlement program and continued working at the same jobs. 
One youth stated that his employer required him to enroll in 
Entitlement or be fired. Thus, Entitlement is paying the 
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salaries of youths which were previously paid by their 
employers . 

In addition to private sector substitution, the issue 
of achieving variation in the subsidy level was not easily 
resolved. As early as the first grant awards in 1978, Labor 
and MDRC required sponsors to submit subsidy reduction plans. 
This requirement was embodied in Entitlement regulations 
issued in October 1979. Subsidy reduction plans were time- 
phased reductions from the 100-percent subsidy levels most 
sponsors were using. Thus, after a youth was assigned to 
a worksite for a particular length of time, 6 or 12 months, 
the private employer was to reimburse the sponsor for part 
of the wages or lose the participants. 

Generally sponsors resisted these attempts, particularly 
Baltimore and Boston. They argued that (1) the 100-percent 
subsidy was necessary to obtain private sector participation 
a -d (2) they had negotiated private sector jobs in good faith 
at the 100-percent level and believed it unfair to renegotiate 
these agreements after the fact. Baltimore finally imple- 
mented a subsidy reduction plan in the fall of 1979. In 
February 1980, MDRC reported that "the plan continues to be 
implemented but without much success." Boston did not imple- 
ment its wage subsidy reduction plan until March 1980. 

Detroit was more successful in implementing subsidy 
reduction. It began implementing its plan in August 1979 
and required employers who had employed Entitlement partici- 
pants for a 6-month period since January 1979 to agree to a 
25-percent subsidy reduction or lose the participant and 
the slot. As of January 1980, this subsidy reduction plan 
affected 118 employers with 249 participants. Only 23 em- 
ployers with 29 participants rejected this plan. 

After much discussion and negotiation, most sponsors 
have not required very many private employers to absorb any 
costs of their Entitlement participants. In this regard, 
MDRC stated in its May 1980 monthly report that subsidy 
reduction plans are having a minimal impact in terms of 
actual dollars collected from private employers. Further, 
it reported that many sponsors are transferring youths to 
other worksites rather than keeping them at their original 
ones at reduced subsidy levels, thus allowing for the con- 
tinuation of '00-percent subsidies. 
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Entitlement was the first large-scale test of the use 
of wage subsidies in the private sector. It is probably 
inevitable that some substitution or displacement would 
occur in a job creation program of this size, particularly 
ere providing a 100-percent subsidy. If, however, subsidies 
to the private sector are to be incorporated in future youth 
initiatives, much more thought must go into the methodology 
for implementing and monitoring their use. 
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CHAPTER 4 

INTENSIVE OVERSIGHT REQUIRED 

TO IMPLEMENT ENTITLEMENT AS A DEMONSTRATION 

In chapters 2 and 3, we discussed Entitlement's opera- 
tional aspects and its program impact. This chapter discusses 
Entitlement as a demonstration. Since Entitlement's results 
might have a major influence on the approach this Nation will 
take in dealing with a problem that has resisted solution, 
it is important that any conclusions regarding the program 
be based on objective information derived from a valid 
demonstration . 

Labor has undertaken a major research effort to answer 
questions raised by the Congress about Entitlement's impact, 
implementation, and cost. From the beginning of Entitlement, 
poor recordkeeping at the sponsor level caused problems in 
providing complete, accurate, and timely data needed for re- 
search. As a result, MDRC made major remedial efforts so 
that the data needed for research can be used. MDRC is con- 
fident that these efforts have resulted in reliable research 
data . 

Labor and MDRC did not allow the demonstration to 
succeed or fail in a normal environment. MDRC had to inter- 
cede in program operations at most Tier I locations and at 
one of the Tier II sites we reviewed. By providing inten- 
sive oversight, monitoring, technical assistance, and in some 
cases direct intervention at a level not normally accorded 
CETA programs. Labor and MDRC were in a unique position to 
have a positive influence on program results. At some spon- 
sors, this special attention greatly improved program opera- 
tions; at others, it salvaged marginal operations. 

THE CONGRESS REQUIRES DETAILED 
INFORMATION ON ENTITLEMENT 

The Congress recognized the need for detailed information 
on Entitlement to aid it in making policy decisions regarding 
future youth employment programs. In this regard, YEDPA 
specifically required that Labor report on the following: 
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1. 



The number of enrolled youths. 



2. The cost of providing jobs. 

3. The degree to which jobs caused out-of-school youths 
to return to school or others to remain in school. 

4. The number of youths provided jobs in relation to 
the total which might have been eligible. 

5. The Tcinds of jobs provided and a description of em- 
ployers . 

6. The degree to which on-the-job or apprenticeship 
training was provided as part of employment. 

7. The estimated cost of such a program if it were to 
be extended to all areas. 

8. The effect jobs had on reducing youth unemployment 
in the areas served by the Entitlement program. 

9. The impact of Entitlement jobs on other job oppor- 
tunities in the area. 

Labor inferred two additional research issues from the Ian- • 
guage of YEDPA: first, the degree to which youths receive a 
quality work experience; and second, the efficacy of differ- 
ent administrative mechanisms in irnplementing Entitlement. 

To answer these issues. Labor engaged MDRC to manage a 
large-scale research effort. This effort seeks to determine 
how well Entitlement accomplishes what it is designed to 
achieve and whether the program should be extended and ex- 
panded nationally. MDRC designed a three-part research 
program to analyze Entitlement's impact, implementation, 
and costs. MDRC subcontracted with Abt Associates, Inc., 
to perform much of the impact analysis. 

MDRC designed the impact analysis to measure partici- 
pation rates of eligible youths and to assess Entitlement's 
impact on labor force participation, employment and earnings, 
and school attendance and completion. Abt is obtaining data 
for this analysis from four waves of interviews with a random 
household sample of eligible youths and their parents in the 
Baltimore, Cincinnati, Denver, and part of Mississippi's 
rural Entitlement areas. Abt is also obtaining data from 
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four matched non-Entitlement locations that will serve as 
control locations. Respectively, the locations are Cleve- 
land, Louisville, Phoenix, and part of Mississippi's non- 
Entitlement area. These control locations match the four 
Entitlement locations in geographic and economic character- 
istics and are being statistically sampled in the same way 
as the Entitlement locations. Abt plans to use the data 
from the control locations to assess the extent to which any 
changes in youths' educational and employment behavior can 
be attributed to Entitlement. 

Using data from the firsu wave of interviews conducted 
in the spring and early summer of 1978, Abt completed an 
analysis of the characteristics of youths eligible for 
Entitlement. To respond to Labor's request for early results 
regarding program impact on school enrollment, Abt and MDRC 
prepared an interim report which provides a preliminary esti- 
mate of Entitlement's early impact on school retention and 
the return to school by dropouts. The report did not provide 
definitive results because it was based on limited data, 
which Abt considered seriously flawed. MDRC expects later 
analyses to provide more definitive results and plans to 
issue the final impact analysis report in 1982. 

MDRC designed the cost analysis to measure program costs 
and estimate the cost of a nationwide Entitlement program. 
MDRC has pointed out to Labor that getting beyond gross cost 
measurements may be impossible due to the nature of the CETA 
funding system and the difficulty in getting detailed timely 
and accurate cost breakdowns from Entitlement locations. For 
the cost analysis, MDRC is relying on data from Entitlement's 
fiscal reporting and information systems. MDRC will use data 
from these systems to determine total program costs, costs 
per participant, and costs per participant year for different 
subgroups of youths . 

In testifying before the Subcommittee on Employment 
Opportunities of the House Education and Labor Committee 
March 1980, an MDRC official provided the following cost 
projections for Entitlement if it is continued through 
fiscal year 1981. Using existing eligibility criteria, 
MDRC estimated 

$85 million to continue at the existing sites, 

$200 million to expand to 20 additional sites 

(1 Tier I and 1 Tier II in each Labor region), 
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— $485 million to expand to all designated poverty 
areas, or 



— $1,173 billion to expand to all eligible youths, 

MDRC designed the third part of the research program, 
the implementation analysis, to describe and assess the 
Entitlement demonstration's operational development at 
various locations. It will attempt to assess the feasibil- 
ity of providing guaranteed jobs and to identify factors 
that affected program operations* The general research on 
Entitlement's implementation covers certain key areas, 
including 

— a description, of program content and operations, 

— implementation factors affecting enrollment and con- 
tent and continued participation, 

— relationships between prime sponsors and schools, 

— the private sector's role in the demonstration, 

— implementing the program in rural areas, 

— Tier II innovations, and 

— the quality of worksites. 

Data for the implementation research come from three primary 
sources; (1) the Entitlement information system, (2) MDRC 
field operations staff, including onsite monitors at Tier I 
locations, and (3) interviews with program officials and 
others knowledgeable of program operations. As of March 
1980, MDRC had issued two interim reports on program imple- 
mentation and costs. 

In addition to the three-part research program, MDRC 
has undertaken special studies which will address worksite 
quality, program implementation in rural areas, private 
sector participation, and labor market effects. In addition 
to a series of reports on specific issues and additional 
interim reports, MDRC expects to issue a final summary 
report on Entitlement in 1982. 



ERIC "^'^ 



42 



^4AJ0R EFFORTS REQUIRED 
TO CORRECT DATA PROBLEMS 

The importance of valid and meaningful data regarding 
the Entitlement demonstration cannot be overemphasized. To 
fulfill their research responsibilities. Labor and MDRC need 
complete, accurate, and timely data on each participant as 
SeTfas information on their Entitlement job. MDRC relies 
on sponsors to provide these data. 

we have not made a detailed review of the data problems 
encountered on the Entitlement demonstration nor are we in a 
position to say whether or not these problems JJ^l^^/^ ^ ^^^^ 
the research results. However, information gathered during 
our review shows that mo-t sponsors experienced problems in 
providing complete, - -urate, and timely Entitlement data 
Seeded for MDRC ' s research, especially for the ^rOf^^^anuo" 
plementation analyses. MDRC is , confident that its cleanup 
efforts have resulted in reliable research data. 

in setting up its research design, MDRC concluded that 
the CETA information system used by sponsors was '^ot suffi- 
cient for the needs of Entitlement. Although the CETA system 
has standardized reporting requirements, it does 2°^^^!,^^ 
standardized forms to collect data. CETA allows sponsors, to 
develop their own forms tailored to their individual needs. 
MDRC needed data on Entitlement participants school and em- 
ployment history, and the CETA system was ^°^.f^^^^ned to 
obtain this. Therefore, MDRC designed an Entitlemex.t mfor 
mation system to collect standardized data among sponsors 
to minimize additional data collection burdens, and to pro- 
vide compatibility with the CETA system. 

As discussed in chapter 1 (see p. 6), the capability of 
a sponsor to meet Entitlement information needs was a ma^or 
cri?erion in the selection process. Labor's grant agreement 
with the chosen sponsors included a copy of the Entitlement 
information system specifications. MDRC P^^P^^f - ^^^^^^ 
instruction manual, distributed it to each participating 
sponsor, and provided a training course on the information 
|ys?em! MDRC continued onsite training activities needed. 
Although it had originally envisioned having onsite MDRC in- 
fon^ation system stiff at each Tier I location, MDRC abandoned 
this plan for cost and efficiency reasons. 
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MDRC has reported that the effectiveness of data collec- 
tion efforts is generally dependent on the following factors: 

"1. the experience, skill, and degree of turnover 
of the local intake, fiscal, and MIS [manage- 
ment information system] staff; 

2 . the intensity of youth enrollment and the 
level of staffing provided to handle the 
flow; 

3. the level of quality control established 
locally to examine documents before their 
submission to MDRC; 

4. the general preparedness of the prime spon- 
sor to handle the data collection function 
smoothly and completely from the onset of the 
demonstration; and 

5. the overall commitment of participating prime 
sponsors to provide high-quality, timely 
data. " 

Poor recordkeeping 

MDRC requires participating sponsors to submit data for 
each participant and job* Supporting files must include 
copies of eligibility records, status change records, and 
various other records which are the supporting documentation 
for the Entitlement information system. After interviewing 
participants, we reviewed their files to determine whether 
they contained the required documentation. Generally they 
did. 

For example, 99 percent of the files contained the 
required enrollment records which document demographic and 
background information about each participant. Also, 99 per- 
cent of the participants' files contained documentation showing 
that the participant was economically disadvantaged, 98 per- 
cent had the documentation indicating that the participant 
had been certified as eligible, and 88 percent contained the 
required documentation showing that the participant was in 
school . 
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We also checked the files to determine whether they 
accurately showed the status of the participants we inter- 
viewed. MDRC requires sponsors to prepare status change 
forms to document three types of status: job assignment, 
hold, and terminations. However, only 68 percent of the 
files accurately recorded a participant's program status. 
In its April 1979 interim implei.ientation report, MDRC re- 
ported problems with participant status records and recog- 
nized that its Entitlement information system data were in- 
complete, inaccurate, and untimely in this regard. 

Certain Tier I sponsors experienced major management 
information and payroll problems requiring the temporary 
suspension of some programs. MDRC, recognizing the lack of 
valid program data, intervened to help resolve the problems. 
More intensive intervention, however, was required at some 
sponsors. (See pp. 49 to 51.) In spite of these efforts, 
the problems persisted at some locations. For example: 

— In August 1978, MDRC found that Baltimore had manage- 
ment information problems which included missing 
forms and participant files, duplicate enrollments, 
the failure to forward status change forms to MDRC, 
a 98-percent error rate 1/ for forms that were for- 
warded, and poor worksite records. MDRC also found 
in August 1978 that Baltimore had conducted little 
management information training for its staff and 
questioned the qualifications of some. Baltimore 
had an information system which MDRC described in 
November 1978 as being "quite slow and error ridden" 
and, therefore, it could not be used for its report 
to Labor and the Congress. In October 1979, MDRC 
criticized the sponsor for slow error correction and 
3 ate submission of performance and fiscal reports. 
In addition, we found that the sponsor often lacked 
participant academic and attendance information. 

According to MDRC, Boston had resolv^sd early informa- 
tion system problems to the extent that MDRC found 
very little, if anything, to criticize and concluded 
that the system seemed to be in excellent shape in 
September 1979. Later, after reviewing payroll 



1/ln commenting on our draft report, Baltimore asserted that 
~ the 98-percent error rate was for 1 week and was not a 
normal condition. 
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records and other information system data, we were 
unable to trace the program history of 19 of 78 par- 
ticipants. We reported to the sponsor that during 
our worksite visits we had interviewed two partici- 
pants whose records showed that they had been ter- 
minated. As a result, Boston . initiated a review of 
4,900 records which revealed that over 1,500 youths 
continued to work after they had been terminated 
from Entitlement. For the period these participants 
worked after termination, terminations were over- 
stated. MDRC requested a certified public accounting 
firm to review the sponsor's internal controls and 
systems flow. At MDRC's insistence, in March 1980 
the sponsor froze the processing of applications and 
transfers to allow staff to concentrate on a massive 
cleanup of the information system. No new applica- 
tions were accepted for 10 weeks during this process. 

— Cincinnati was consistently delinquent in submitting 
required data.. At one point, it was 10 months behind 
schedule in submitting monthly performance reports, 
and MDRC described the one participant roster that 
had been submitted as so incomplete that it was use- 
less. The sponsor had to revise its monthly operat- 
ing reports for Apri.i 1978 to March 1979 due to ex- 
penditure discrepancies. However, even with the 
revisions, we had difficulty reconciling the data 
from one report to the next. 

While each of these examples related to a particular 
Tier I sponsor, management information problems were wide- 
spread. For example, each Tier I and some Tier II sponsors 
had problems in obtaining and submitting data on school 
attendance. Also, as of November 1979, MDRC's Entitlement 
information data showed that youths assigned to a job and 
not working ranged from 10 to 47 percent, averaging 23 per- 
cent for the seven Tier I prime sponsors. We recognize that 
there may be legitimate reasons why assigned youths would 
not be working, such as work absences, but if they are not 
working for extended periods, the sponsor should place them 
in a hold or terminated status. 

After the completion of our fieldwork, MDRC officials 
stated that, in spite of the recordkeeping problems at the 
sponsor level, they were confident the data were sufficiently 
reliable for research purposes, MDRC told us that, in sum- 
mary, it has three distinct levels of quality control: first, 
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it checks each document and report before entering it into 
the computer; second, it identifies errors of logic; and 
third, it reconciles EntitlenK^nt information system data with 
fiscal reports and telephone call-in data. In addition, MDRC 
also conducts periodic reviews and requires corrective action. 

ENTITLEMENT WAS NOT A 
REAL WORLD DEMONSTRATION 

Labor regulations state that the basic purpose of En- 
titlement Tier I projects is to test the experimental idea 
of a guaranteed job by determining whether sponsors can fea- 
sibly implement substantial programs. Tier II. projects are 
to test specific Entitlement innovations. To be a valid 
demonstration, we believe sponsors should conduct the pro- 
gram in a normal environment, that is, one similar to that 
in which they operate other CETA programs . Labor and MDRC 
did not permit a real world test of Entitlement's success 
or failure. Due to widespread problems, they interceded to 
salvage some marginal operations by providing intensive over- 
sight, monitoring, and technical assistance. Only rarely, 
however, are sponsors accorded the degree of oversight, mon- 
itoring, and technical assistance received by the Entitle- 
ment sponsors. 

Normally, Labor regional representatives assigned to 
each sponsor are the focal point through which the spon- 
sors' effectiveness is evaluated. Their monitoring activ- 
ities for all CETA programs operated by that sponsor «pon- 
sist of (1) routine contacts with sponsor officials either 
by telephone or occasional site visits, (2) desk reviews and 
evaluations of quarterly reports, (3) an annual assessment 
of sponsor performance, and (4) reviews to determine whether 
sponsors are complying with CETA's requirements. Time and 
workload constraints do not permit Labor regional representa- 
tives to continuously monitor programs in the manner that 
MDRC monitored Entitlement. 

MDRC has given Entitlement sponsors more centralized 
direction and closer monitoring than Labor normally provides 
for other CETA programs. As Labor's guiding agency for 
Entitlement, MDRC has certain operational responsibilities 
in addition to its research duties. To guide and assess the 
performance of sponsors from both operational and research 
standpoints. Labor required that MDRC establish continuous 
communication with the local Entitlement projects. 
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Through its field operations staff and onsite monitors 
at each Tier I location, MDRC monitors the Entitlement proj- 
ects to see that they fulfill their programmatic, fiscal, 
and legal responsibilities. MDRC also monitors to see that 
the projects operate consistently with the overall objectives 
of YEDPA and the program design. MDRC field and onsite staff 
prepare periodic reports on operations at each project. 
Additional responsibilities include the identification and 
assessment of prime sponsor training and technical assistance 
needs. In this regard, MDRC is responsible for identifying 
and recommending consultants when needed and chahneling use- 
ful operational information to the Entitlement sponsors. 

Program reviews 

During the summer of 1978, MDRC became aware of tne prob- 
lems in managing a large-scale demonstration. The summer 
months gave Entitlement sponsors their first real operational 
test because, in addition to their Entitlement responsibili- 
ties, they were also responsible for implementing the Summer 
Youth Employment Program. In some locations, sponsors oper- 
ated the programs in an integrated fashion. 

MDRC records indicate that problems erupted at almost all 
Tier I and some of the larger Tier II locations. MDRC re- 
sponded by sending teams to the sponsors to make firsthand 
program reviews. Typically, the teams consisted of Labor 
representatives and MDRC senior level, field operations, 
information system, and onsite staff. 

The scope of the review was a comprehensive walk-through 
of the orogram, component by component, to help the team 
understand the program's operations. The program reviews 
caused some sponsors, who were unaccustomed to such close 
examination, to assess and redirect operations. The spon- 
sors. Labor, and MDRC agreed that such reviews were an effec- 
tive management tool. As a result, program reviews became a 
regular occurrence with a specified format. Each Tier I 
sponsor has had at least two reviews and over half of the 
Tier II sponsors have had reviews. 

MDRC designed the program reviews as a means to assess 
management and operational capability and performance, to 
identify specific problem areas,., and to recommend any neces- 
sary corrective measures. In addition, the reviews include 
an assessment of the management and operation of the infor- 
mation and fiscal systems. Tier II program reviews also 
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include an assessment of the program's innovative feature. 
The reviews are comprehensive and include sponsor presenta- 
tions on recruitment, intake, worksite development and moni- 
toring; monitoring and enforcement of academic and attendance 
standards; and payroll, fiscal, budget, and information 
system management. Team members review participants' files, 
performance reports, enrollment forms, and other information 
system records. They interview program staff and visit work- 
sites, intake centers, and alternative education programs or 
schools. The team concludes the program review by providing 
the sponsor with feedback on findings and recommended correc- 
tive actions. Often followup visits are made to see what 
progress a sponsor has made as a result of the review. 

We agree that these comprehensive and intensive program 
reviews are an effective and needed management tool and they 
are indicative of the degree of oversight necessary to main- 
tain quality youth programs. Without the intense oversight 
provided by these reviews, it is difficult to state what the 
fate of Entitlement would have been at some locations. The 
following examples illustrate positive impacts of reviews 
which may not have resulted in a normal program environment. 

— Shortly after program operations began, Denver encoun- 
tered problems which persisted. Early in the program, 
the use of incorrect income guidelines resulted in 
enrolling many ineligible youths. Many errors on 
enrollment forms were not corrected because it would 
have required reinterviewing 2,000 youths. The pro- 
gram received negative media attention due to enroll- 
ing the ineligible youths, alleged payroll fraud, and 
the few participants placed in jobs. In April 1978, 
only 200 of 1,200 enrolled youths were assigned to 
jobs. Original projected average enrollment was 3,550. 
Staff turnover was high, including the program director 
position. A poor relationship between the sponsor and 
city agencies affected program operations, particularly 
payroll. Relationships between the sponsor and most 
contractors who were responsible for significant areas 
of program operation deteriorated. 

MDRC, with Labor's concurrence, became directly in- 
volved in managing the Denver Entitlement program. 
Because the sponsor encountered difficulty in hiring 
needed employees, MDRC agreed to pay the wages for 
some staff to perform monitoring, management informa- 
tion, and other functions. MDRC allowed its cnsite 
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monitor to work closely with sponsor staff in all 
areas of program activity and report frequently to 
MDRC in New York. MDRC prepared and conducted a 
training session for Denver administrative staff. 
To establish a good solid program on a smaller scale, 
Denver, at the insistence of MDRC and Labor, stopped 
enrolling new participants on June 30, 1979. As of 
June 1980, new enrollments were still prohibited. 

On the basis of a January 1980 review, MDRC made major 
program and administrative recommendations which Denver 
adopted. The sponsor obtained new contractors to per- 
form Entitlement functions and established control mech- 
anisms for monitoring contractor performance. A public 
accounting firm assessed the payroll system and recom- 
mended improvements . Although some improvements were 
apparently made, MDRC saw a potential danger in the 
sponsor's continuing inability to maintain control over 
payroll operations and management information. MDRC 
recognized the need to continue watching the situation 
carefully because of the research need for accurate data 
and controls on data collection. In spite of these con- 
cerns, MDRC believes that operations have stabilized 
and the program is moving forward, although at a much 
smaller level than planned (about 400 paid participants). 

— Detroit also encountered problems soon after operations 
began. During the early months of the program, about 
30 percent of the participants waited 8 weeks or longer 
to get paid. Many of these youths had been assigned 
to worksites before the data necessary to put them in 
a pay status were prepared. As a result, the payroll 
system rejected their timesheets. By May 1978, less 
than half of the 2,200 enrolled youths had been as- 
signed jobs. Administrative staff circumvented basic 
program systems to increase job assignments. There- 
fore, many participants' files and records were lost 
or left incomplete; Entitlement information system 
problems arose; and relations between the prime spon- 
sor and its major contractor — tl\e school system — 
deteriorated. Labor and MDRC pressed for resolution 
of these problems and ordered a shutdown from Septem- 
ber to November 1978 to allow time for their resolu- 
tion. During this period the participants were 
laid off. The sponsor terminated its relationship 
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v/ith the Eschool system as the managing agent and 
assumed the role itself. 



A public accounting firm audit of the payroll system 
in June 1979 resulted in many recommendations, but 
problems continued with missing, incomplete, or er- 
roneous payroll data. In addition, Detroit continued 
to have management information problems. Its data 
showed a significant discrepancy between the number 
of participants enrolled and actually working, pri- 
marily because the sponsor continued to list many 
youths as enrolled when they were no longer active in 
the program. At one time, the discrepancy exceeded 
3,000 youths. In December 1979, only 50 percent of 
the youths that the sponsor listed as enrolled re- 
ceived a paycheck. After reviewing 700 files, the 
sponsor found that 400 of these youths should have 
been terminated; others had been terminated but not 
removed from the payroll; and still others needed a 
job reassignment. Despite these problems, the con- 
tinued involvement of MDRC and the assignment of more 
qualified staff resulted in significant program im- 
provements . 

— Almost since program inception. Entitlement in Dayton 
suffered from three interrelated problems: low enroll- 
ment, an unsuccessful on-the-job training component 
(which was the program's innovative feature), and high 
management costs. MDRC provided technical assistance 
by offering suggestions and recommendations on how 
Dayton should ove-rcome its problems. In fact, MDRC 
gave the sponsor and its major contractor responsi- 
bility for taking specific corrective actions with 
deadlines. MDRC suggested new recruitment strategies 
and told the sponsor to explain how a contract change 
would result in increased on-the-job training partici- 
pation. MDRC also required a plan to improve the 
participant/program management cost ratio from 40/60 
to the approved 67/33. MDRC allowed Dayton to expand 
its recruitment territory. The sponsor believed this 
was the only way to resolve its cost ratio problem. 
Enrollment has now increased from less than 50 to over 
200 after expansion. 
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CHAPTER 5 



CONCLUSIONS AND REC0^4MENDATI0NS 

CONCLUSIONS 

After over 2 years of operating Entitlement, serious 
questions exist as to whether CETA prime sponsors have the 
capacity to effectively implement a program of this magnitude 
in a short time. Intensive monitoring and oversight efforts 
were implemented by Labor and MDRC to deal with a litany of 
serious operational problems. These efforts met with mixed 
results . Some problems were caused by rapid program startup 
and were resolved during the second year, other problems 
were more persistent and their solution more evasive, and 
some new problems surfaced during the second year. There- 
fore, a gradual approach to dealing with the youth unemploy- 
ment problem will be advantageous in the long run because 
sponsors would have the opportunity to develop the management 
capacity needed to implement large-scale programs. 

Operational problems were so severe at some locations 
that it is difficult to tell how the program would have been 
implemented had effective management been employed by the 
sponsors. Although Labor and MDRC established goals and 
regulations to implement the legislation, sponsors had diffi- 
culty complying with the regulations and meeting the goals. 

The question of Labor and MDRC intervention and its im- 
pact on the Entitlement demonstration represents a dilemma 
in implementing programs of this type. If the demonstration 
had been allowed to proceed in the normal CETA environment 
without intensive oversight, the program might have failed 
because of its severe operational difficulties. On the other 
hand, the degree of monitoring and direct intervention that 
took place shows what is needed to operate a complex youth 
employment program but may not be practical to implement from 
a financial or management standpoint. 

If Entitlement is to be extended, expanded, or a similar 
program undertaken, certain areas need intensive management 
attention . 
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— First, it appears that tying jobs to meeting school 
academic and attendance standards is a valid concept. 
However, sponsors and schools should agree on meaning- 
ful standards, and administrative processes need to be 
established to enforce them fairly. 

— Second, relationships among sponsors and other organi- 
zations that help youths need to be upgraded. Prob- 
lems among these organizations now often adversely 
affect youths. 

— Third, most worksites provided an adequate work ex- 
perience but more effort is needed to improve supervi- 
sion, participant counseling, job development, and 
worksite monitoring. 

— Fourth, Entitlement is the first youth employment pro- 
gram which made extensive use of subsidized private 
sector jobs. To the extent possible, attention must 
now be directed to assuring that jobs subsidized by 
the Federal Government, particularly at or near 
100 percent, are in addition to jobs that normally 
would be made available to youths by these employers. 

— Fifth, the Entitlement concept provides for a guaran- 
teed job; therefore, if it is expanded the availabil- 
ity of funds to pay participants ' wages might depend 
on controlling program management costs. 

RECOMMENDATION TO THE CONGRESS 



New programs, particularly a complex demonstration such 
as Entitlement, inevitably face implementation problems. If 
Entitlement is to be extended or used as the basis for future 
youth initiatives, we believe that the Congress should recog- 
nize the conditions under which the demonstration was imple- 
mented. Some competitively selected sponsors receiving inten- 
sive oversight had significant operational problems. If the 
Entitlement concept is to be expanded to other sponsors or a 
similar program adopted, we recommend that the Congress as- 
sure that it be implemented gradually to allow sponsors time 
to develop the necessary management capacity. 
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RECOMMENDATIONS TO THE 
SECRETARY OF LABOR 

If Entitlement is extended or expanded, or a similar pro- 
gram is adopted, we recoiumend that the Secretary 

— require sponsors to strictly enforce reasonable 
attendance and academic standards thereby enhancing 
the program's credibility and effectiveness for both 
participants and the general public ; 

— increase efforts to promott; cooperative relationships 
between sponsors, schools, and other organizations 
serving youths? 

— help sponsors upgrade worksite supervision, job 
development, participant counseling, and sponsor 
monitoring; 

— create, implement, and enforce adequate controls to 
prevent substitution; and 

— establish and enforce requirements that limit program 
management costs. 

MDRC AND SPONSOR COMMENTS 
AND OUR EVALUATION 

We requested comments from Labor, MDRC, and the 11 spon- 
sors included in our review. MDRC ' s response and responsesi 
from the Albuquerque, New Mexico; Baltimore, Maryland; Hills- 
borough County, New Hampshire; Philadelphia, Pennsylvania; 
and Mississippi sponsors are included in appendixes IV through 
IX. After the 30-day comment period provided by Public Law 
96-226 expired. Labor submitted comments which concurred with 
all of our recommendations. Labor's comments, which are in- 
cluded as appendix X, have not been analyzed in this report. 

MDRC and Baltimore disagreed with our approach to this 
study. 

MDRC 

MDRC commented that our specific findings do not sub- 
stantially vary from many of theirs. MDRC did, however, 
express some concern vzith our draft report. It did not 
believe that the report sufficiently distinguished between 
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the structure of a demonstration and an operating program. 
MDRC believed we failed to convey that the primary mission 
of the demonstration was to inform policymakers about the 
feasibility and implications of the Entitlement concept and 
its component parts. MDRC pointed out that the demonstra- 
tion was to yield information on whether the program should 
be continued, expanded, reworked, or dropped as well as to 
help in the design of other youth programs. 

We recognize that Entitlement is a demonstration and 
refer to it as such throughout the report. Furthermore, 
we discuss Entitlement as a demonstration extensively in 
chapters 1 and 4. We agree that Entitlement was intended 
to provide information to aid in the design of other youth- 
related efforts, and it is on the basis of information 
gathered during our review that we have made our recommen- 
dations to the Congress. 

By placing too much emphasis on the demonstration aspect 
of Entitlement, however, MDRC seems to lose sight of the im- 
portance of the program's operational aspect. Much of the 
evaluation of Entitlement as a demonstration is dependent on 
its operational successes and failures. For example, MDRC 
stated that it is not appropriate to say that the sponsors 
failed to adequately enforce educational standards. Instead 
MDRC sair" the focus should be on what could be achieved given 
the vagaries of local educational standards. The enforcement 
of educational standards, however, was not a matter of choice. 
It was specifically provided for in YEDPA. It is clear that 
sponsors had great difficulty, or were unable, to comply with 
the requirement to enforce educational standards. Thus, in 
any future programs, more emphasis must be placed on solving 
the enforcement problem if the educational requirements are 
retained. 

MDRC also stated that difficulties in the rapid startup 
should be looked upon as a learning process and that sponsors 
which changed their management structures should be credited 
rather than criticized for their willingness and ability to 
make changes. We did not criticize sponsors for their will- 
ingness or ability to change management structures. Rather, 
we reported the conditions which led to the changes. The key 
lesson learned from the rapid startup forms the basis for our 
recommendation to the Congress. 
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MDRC did not believe we recognized sufficiently that a 
basic purpose of tbe demonstration was to assess its long- 
term employment and educational impact. It pointed out that, 
depending on the results of the impact research, the problems 
described in the report may not be significant. MDRC also 
stated that the level of supervision and control it provided 
was necessary to assure that the demonstration yielded the 
needed information regarding longer term impacts. 

We recognize the importance of Entitlement's research 
of the impact on youths, which is described in chapter 4. 
Obviously, since data were not yet available, we could not 
consider the results of that research. However, we believe 
that, regardless of the outcome of the impact analysis, im- 
proved operations v/ill translate into improved results. We 
agree that MDRC ' s high level of supervision and control was 
necessary to obtain the needed information from Entitlement. 
However, it is not likely to be replicated by Labor in an 
operational rather than demonstration setting. 

MDRC Questioned our evaluation because it lacks stand- 
ards of performance. MDRC stated that the report is slanted 
toward the negative, thereby leaving the reader with the im- 
pression of failure. First, we would like to point out that 
both positive and negative examples were included. Second, 
we discussed the subject of standards, particularly for work- 
sites, with MDRC officials at great length during our exit 
conference. They agreed that specific standards of this 
nature do not exist. In the absence of standards, the cri- 
teria that we used are spelled out in the report. 

MDRC mentioned the issue of job development and sponsor 
links with other organizations as examples of unbalanced 
presentations. We disagree. Regarding job development, we 
state that the sponsors initially appeared to have developed 
enough jobs. Most Tier I sponsors, however, had difficulty 
creating or maintaining an adequate number of part-time jobs 
during the school year. We also point out the administrative 
difficulties some sponsors had in assigning youths to jobs m 
a timely manner. We support these statements with examples 
from three sponsors. Furthermore, MDRC stated that Entitle- 
ment found jobs for 80,000 youths in a relatively short time. 
The number of youths assigned to jobs at any one time, how- 
ever, never exceeded 30,000. 
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Regarding the linkage issue, we credit all sponsors we 
visited with making some progress. On the other hand, we 
state that a number of important relationships were not 
always lasting or without conflict. We then describe some 
of the problems, their causes, and where appropriate, the 
action taken to resolve them. 

MDRC pointed out several instances where it claims the 
report deals less than adequately with certain issues. 

First, it stated that intensive, individual counseling 
was not a demonstration objective, and it is unfair to imply 
a failure to achieve this as a program goal. We have not 
stated that intensive, individual counseling was an ob:iective 
of the demonstration. We did point out, however, that ac- 
cording to their grant applications, which became part of the 
grant agreements, all Tier I sponsors planned to provide at 
least some counseling. We believe that their plans to pro- 
vide counseling and the high negative termination rate indi- 
cate that additional counseling would have been advantageous 
to many youths . 

Second, MDRC stated that our use of the term "program 
management costs" is misleading, MDRC, in its reply to our 
draft report, separated program management costs from client 
service costs and stated that program management costs aver- 
aged less than 20 percent. We used the Entitlement program 
definition of program management costs contained m official 
MDRC instructions to the sponsors which combined program ad 
ministration with other categories, such as client services 
and worksite supervision. Regardless of how the costs are 
classified, they are higher than anticipated. 

MDRC further stated that our discussion of underenroll- 
ment implies underachievement . We do not believe that under- 
enrollment necessarily implies underachievement. We Jimply 
point out that enrollment was less than the sponsors had 
estimated. Our discussion of the underenrollment issue is 
not a criticism of MDRC or sponsor operations. Instead, 
underenrollment is identified as a major cause of the high 
management cost rates . 

Third, MDRC stated that the section of the report on 
displacement is weak. MDRC pointed out that the report 
stated that an indepth review of this issue was not conducted, 
and that we concluded that substitution occurred m 45 percent 
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of the private sector jobs. We did not say that substitution 
or displacement occurred in 45 percent of the private sector 
jobs. We state that Labor and the sponsors need to monitor 
the payment of wage subsidies because about 45 percent of 
the private sector jobs were with employers who normally em- 
ploy youths . 

MDRC also pointed out that a major study of the dis- 
placement issue is being conducted at the behest of Labor 
and the Office of Management and Budget. After receipt of 
MDRC's comments, however, we learned that the study, which 
was being conducted by two MDRC contractors, was suspended 
because its complexity resulted in considerable cost overruns. 

Fourth, MDRC stated that we confused short-term data 
problems of the sponsors with the. ultimate reliability of 
the data for research and reporting purposes. MDRC also 
stated that we misinterpreted the quality control require- 
ments, giving them a negative implication. MDRC included an 
attachment to its comments more fully describing its Entitle- 
ment Information System. In our discussion of recordkeeping, 
we did not confuse the sponsors' data problems with the ulti- 
mate reliability of the data after they have gone through 
MDRC ' s quality control procedures. We specifically state 
that, in spite of recordkeeping problems at the sites, MDRC 
officials were confident that the data v/ere sufficiently re- 
liable for research purposes. Although we believe that we 
have sufficiently summarized MDRC ' s Entitlement Information 
System and quality control procedures, we have included MDRC ' s 
description in appendix IV. 

Sponsors 

Baltimore stated that we did not give clear recognition 
to some valuable lessons learned from the demonstration. 
First, Baltimore stated that the demonstration was, in part, 
designed to determine whether large numbers of youths would 
be willing to work if given the opportunity. Baltimore con- 
cluded that the demonstration proved that they would. 

While we do not disagree, we do believe that it is sig- 
nificant to point out that MDRC estimated that enrollment 
levels in the first year were approximately 40 percent of 
the total eligible population, and in comments to us, MDRC 
estimated 50 percent participation in 1980. This indicates 
that a large number of youths did not take advantage of the 
Entitlement opportunity. 
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Second. Baltimore stated that the demonstration was ^ 
successful in involving the private sector in providing jobs 
for youths. "We agree that the demonstration did attract sub- 
stantial private sector involvement. However, for much of 
the program, private sector jobs were 100 percent federally 
subsidized, and a significant percentage were the type that 
youths would normally obtain. 

Third, Baltimore stated that the demonstration shows 
that sponsors could develop the management capacity to 
operate large-scale employment programs. vniile we agree 
that some sponsors may have eventually developed the manage- 
ment capacity to operate large programs, others have not. 
Chapter 3 discusses the operational problems at length. 
Further, our recommendation to the Congress recognizes that 
sponsors need time and assistance to develop the management 
capacity necessary to operate large-scale, quality employ- 
ment programs . 

In addition to its general comments, Baltimore raised a 
number of specific points. Baltimore stated that the report 
is quite unbalanced and takes issue with the technique of 
stating a general conclusion and using anecdotal instances 
from one or two sites rather than specific data. Second, it 
maintained that all sites had both positive and negative ex- 
periences, and that we did not maintain balance m selecting 
our examples. To do so, Baltimore suggested, invites un- 
warranted generalizations and unfavorable conclusions. 

We do not agree. Chapter 2 clearly credits sponsors 
with providing adequate work experiences at most worksites. 
Chapter 1 clearly describes the statistical sampling techni- 
ques used to select worksites. Examples of worksites at 
specific locations were chosen as representative of positive 
and negative aspects of the demonstration as a whole. We 
did not compare the quality of worksites among sponsors be- 
cause our sampling plan only allowed us to project to the 
universe of Tier I worksites. 

We have provided a proper balance among the sponsors, 
where warranted, in describing the operational problems in 
chapter 3 and the data problems in chapter 4. We do not 
believe we have singled out any particular sponsor as the 
Baltimore comjnents appear to assert. 
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Baltimore asserted that we failed to recognize the value 
of change. Again, we disagree. We believe that we recog- 
nize the value of change where it led to improvements. For 
example, we report that Baltimore took corrective action to 
remedy some worksite and paperwork problems. 

Baltimore also stated that we made a number of factual 
or interpretational errors. Citing our discussion of ter- 
minations, Baltimore stated that we failed to recognize job 
placements as positive terminations. In our analysis, we 
used MDRC termination statistics which classify job place- 
ments as a neutral outcome for Entitlement participants. We 
agree with MDRC's position since the immediate goal of En- 
titlement was not to help youths move to jobs outside the 
program but to help them obtain a high school diploma or 
its equivalent. 

Baltimore stated that we attributed the fact that it 
had the lowest cost per participant to the lower number of 
hours worked per participant. Baltimore claimed that, if 
adjustment is made for the lower hours, attendance rates and 
breaks, it still had the lowest cost per participant. While 
other factors may affect the cost per participant, we believe 
the number of hours is the major factor in Baltimore's cost 
being lowest. Furthermore, Baltimore provided no support 
for its statement that it had the lowest cost per participant 
after these factors were considered. 

Baltimore stated that we have taken data from several 
MDRC sources, misquoted the data, and rearranged them in a 
manner that changed the original MDRC intent. We have re- 
viewed the source documents relative to the example Balti- 
more' s comments cited regarding its information system. The 
basic facts remain unchanged; however, we have made several 
minor changes to give the reader a better perspective of the 
problem. While the 98-percent error rate mentioned by Balti- 
more may be an extreme situation, the fact still remains that 
the accuracy and timeliness of its information system data 
remained a problem as late as October 1979. 

Philadelphia stated it was in general agreement with the 
major findings in the areas of operational problems, worksite 
experiences, and the test of Entitlement as a guaranteed job 
program. Philadelphia mentioned that three specific com- 
ponents of its Entitlement program deserved replication: 
private sector worksite development, clerical skills enrich- 
ment, and concentration of services in a targeted area. 
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Although we did not specifically review these aspects in 
Philadelphia, we agree that successful programs should be 
continued and expanded . 

Philadelphia, in commenting on MDRC ' s role in the demon- 
stration, stated that many operational and administrative 
constraints that have occurred can be attributed to MDRC 
and do not occur between the sponsor and the Labor regional 
office. Philadelphia urged us to include a section m the 
report on MDRC's role as the program manager. We have de- 
scribed MDRC's role in chapters 1 and 4 . We understand why 
the sponsor would have a different relationship with the 
Labor regional office than it had with MDRC since MDRC pro- 
vided Entitlement sponsors with more centralized direction 
and closer monitoring than Labor normally provides for other 
CETA programs . 

Mississippi concurred with our major findings and sup- 
ported our recommendations to the Congress and the Secretary 
of Labor. Mississippi did note a few areas needing further- 
clarification. First, Mississippi stated that its monitoring 
efforts were much more thorough than indicated in our report. 
We have expanded our discussion of Mississippi's monitoring 
to recognize this effort. 

Mississippi also said that MDRC's concern in March 1980 
that sponsors were reluctant to terminate youths and were 
slow to enforce standards did not apply to its current pro- 
gram. Mississippi said that clearly defined academic and 
attendance were established about 6 months ago and have, been 
enforced since that time. In our view, this should enhance 
Mississippi's program. 

In reference to our statement on the low priority spon- 
sors placed on counseling, Mississippi stated that counselors 
were restricted on the amount of time that could be spent with 
participants, because the participants were either in class or 
at work. It stated, however, that more counseling is needed 
and should be considered in the design of any related program. 
We agree and have recommended that efforts be made to upgrade 
counseling . 

Albuquerque stated that the report was a fair represen- 
tation of Entitlement. Hillsborough County had no specific 
comments on the report. 
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RECENT GAP REPORTS O N 
Y OUTH EMPLOYMENT AND TRAINING PROGRAMS 

"Payment Problems in tlie Summer Youth Employment Program 
in New York City" (HRD-77-18, Feb. 2, 1977). 

"Information on the Summer Youth Employment Program" 
(HRD-77-121, June 27, 1977). 

"Poor Administration of the 1977 Summer Program for Econo- 
mically Disadvantaged Youth in New York City" (HRD-78-123, 
July 26, 1978) . 

"More Effective Management Is Needed to Improve the Quality 
of the Summer Youth Employment Program" (HRD-79-45, Feb. 20, 
1979) . 

"Job Corps Should Strengthen Eligibility Requirements and 
Fully Disclose Performance" (HRD-79-60, July 9, 1979). 

"Department of Labor's Efforts to Improve the Quality of 
the 1979 Summer Youth Employment Program" (HRD-80-39, 
Feb. 5 , 1980) . 
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PRIME SPONSORS SELECTED FOR PARTICIPATION 
ILT ENTITLEMENT DEMONSTRATION 



Tier I 



a/l. Baltimore Metropolitan Manpower Consortium, 

~~ Baltimore, Md . 

a/2. City of Boston, Boston, Mass. 

a/3. City of Cincinnati, Cincinnati, Ohio. 

a/4. City and County of Denver, Denver, Colo. 

a/5. City of Detroit, Detroit, Mich. 

a/6. King-Snohomisti Manpower Consortium, Seattle, 

~ Wash . 

a/7. Balance of State Mississippi/ Jackson, Miss. 



Tier II 

1. Alachua County, Gainesville, Fla. 

a/2. City of Albuquerque, Albuquerque, N.Mex. 

~ 3. City of Berkeley, Berkeley, Calif. 

a/4. Hillsborough County, Manchester, N.H. 

a/5. Miami Valley Manpower Consortium, Dayton, Ohio 

~ 5. county of Monterey, Salinas, Calif. 

7. New York City, New York, N.Y. 

a/8. City of Philadelphia, Philadelphia, Pa- 

9. Steuben County, Bath, N.Y. 

10. City of Syracuse, Syracuse, N.Y. 

a/lncluded in GAO review. 
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SCHOOL ACADEMIC AND ATTENDANCE STANDARDS FOR 1979 



TIER I 



Sponsor 
Baltimore 



Boston 



Cincinnati 



Denver 



Detroit 



King^ 

Snohomish 



Mississippi 



Academic 



H.S-: 60 average H.S. 



GED: satisfactory GED: 
progress 

H.S.: passing grades HOG- 



GED : satisfactory GED; 
progress 

H-S. : "D'* average H-S. 



GED: satisfactory GED: 
progress 

H-S.: satisfactory perform- H*S- 
ance in at least 2 
out of 3 subjects 

GED: satisfactory GED: 
progress 

H.S.: passing grades in H.S. 
3 subjects 



GED: satisfactory GED: 
progress 

H,S-: passing grades in H-S- 
one subject ("D" 
average ) 

GED: passing grades in GED: 
2 subjects 

H-S. : passing at least H.S. 
2 subjects ( com- 
mitee review of 
individual cases ) 

GED: satisfactory GEDj 
progress 



Attendance 

no more than 4 un- 
excused absences 
per month 

( none ) 



no more than 25 
percent unexcused 
absences 

( none ) 



no more than 25 
percent unexcused 
absences 

( none ) 

no more than 5 un- 
excused absences 
per s emester 

( none ) 

no more than 5 un- 
excused absences 
per semester 

( none ) 



varies with 
each district 



( none ) 



varies with each 
district 



no more than 5 
hours of unexcused 
absence from class 
per month 
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Sponsor 
Alachua County 



Albuquerque 



Berkeley 



Dayton 



Hil Isborough 



Monterey 



New York 



Philadelphia 



Steuben County 



Syracuse 



H . S . 



GED: 
H .S- r 

GED: 
H.S. : 

GED: 
H .8. : 

GED: 
H • S • t 

GED: 
H .S . s 

GED: 

H.S- : 

GED: 
H.S. : 

GED: 
H.S. : 

GED: 

H.S. 

GED: 



TIER II 

Academic 

"D" in at least H.S. 
4 subjects 



satisfactory progress GED: 

passing grades in H,S,; 
3 subjects 

satisfactory progress GED: 

"C" average H.S. 



satisfactory progress GED: 

pass i ng g rades in H - S . 
4 or 5 subjects 

satisfactory GED: 

(none recorded) H.S. 

.satisfactory progress GED: 

passing grades in H.S. 
4 subjects 

satisfactory progress GED: 



65 average in at H,S. 
least 2 subjects 



satisfactory progress GED: 

"D" average H.S. 

satisfactory progress GED: 

6 5 average H . S - 

satisfactory progress GED: 



pass ing grades i n 

80 percent of courses 



Attendance 

Hawthorne: no more than 15 
unexcused absences per semes- 
ter. Eastside: no more than 
5 unexcused absences per 
grad ing period 

( none ) 

no more than 5 unexcused 
absences per quarter 

( none ) 

more than 3 unexcused 
absences in a 6-week period 
results in a conference with 
counselor 

( none ) 

no more than 15 percent 
unexcused absences 

75 percent attendance 

no more than 7 absences 
per semester 

( none ) 

no more than 2 unexcused 
absences per semester 

must attend at least 4 
hours per week 

no more chan 5 consecutive 
unexcused absences — no more 
than 3 discrepancies between 
school and work attendance 

( none ) 

no more than 8 absences 
for report period 

( none ) 

absence from school means 
not allowed to workktthat day 

must attend at least 6 
hours per week 



H.S.: BO percent attendance 



satisfactory progress GED: 



( none ) 



Notes: The standards shown are for the 1978-79 school year, and represent levels of 
performance and attendance required of youths in the Entitlement Demonstra- 
tion. They do not necessarily correspond to the standards for satisfactory 
performance applicable to all school youths. 

Source: MDRC, March 1980, Second Interinj Report on Program Implementation. 
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Bfinrd nf Dirt don: 
Eii Ciinzbrr/i 

Chatrman 
HobrrI Solow 

Vicr' Chairman 
Richard S a than 

Brmard Anderwo^ 
Anthony Dou'tim 
M. Carl Holman 
Alan KUtier 
David Schuitr 
Gilbert Striner 
PhyllU Walla-e 
San ^''atrrman 

Wiitiam J, Ohnker 
Pretidmt 



Mr. Gregory J< Ahart 
Director 

Human Resources Division 
U.S. General Accounting Office 
441 G Street, N.W. 
Washington, D.C. 20548 

Dear Mr. Ahart: 

T^is is in reply to your letter dated Auaust 21, 1980 in 
which you offered us an opportunity to respond formally to the 
GAO draft report on the Youth Entitlement Pilot Projects. I 
would initially note that members of the Board and staff of MDRC 
have, on several occasions, had the benefit of discussing with 
GAO officials the scope, general nature and findings of this 
report. As we have stated previously, there are certain areas 
in whicrh we disagree with the approach taken by GAO in this 
study. 

These break down into four primary issues of concern. First, 
we believe that the report does not sufficiently distinguish between 
the structure of a demonstration and an operating program. Second, 
we are concerned that the report does not effectively recognize 
the long term goals of the research. Third, we believe that the 
report fails to come to grips with the issue of how standards 
should be applied in making judgments about program effectiveness. 
Last, we feel there are several substantive areas raised in the 
report that are not dealt with adequately. I will briefly treat 
each of these issues in turn. 

1. We believe one significant problem is the failure of 
the report to convey the fact that this was a demonstration whose 
primary mission was to inform policymakers on the feasibility and 
implications of the overall Entitlement concept and its various 
coiuponent parts. Perhaps the most cogent statement of the purposes 
^ f Entitlement appears in the Report of the House ISducation and 
Labor Committee on the Youth Act of 1980 (H.R.6711), recently 
approved by the House. In making its recommendations on Entitlement 
the Committer stated: 
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unlike many other Federal programs, the Coiranittee 
did not harbor any preconceived notions or 
heightened expectations- All the Committee desired 
was enough information to determine whether the 
program should be continued, expanded, reworked, 
or dropped all together. The Committee hoped that 
the research would help in the design in other related 
youth efforts. 

GAO's work in distinction to the above purpose generally treats 
Entitlement as an operating program rather than a demonstration, designed 
to be a learning experience. For example, GAO is critical of the general 
nroblems incurred by the sites in establishing and enforcing educational 
standards. This was, indeed, a problem for individual sites because 
within local education systems such standards were either non-existent or 
so vague and varied as to be useless as enforcement tools. The matter of 
educational standards is, in addition, traditionally one of local respon- 
sibility rather than Federal authority. It is therefore not appropriate, 
in our view, to say that the prime sponsors "failed" to perform adequately 
on this facet of the demonstration. Rather the focus should be on what 
realistically could be achieved, given the vagaries of local educational 
standards so tl at policymakers could adjust their aspirations and expec- 
tations accordingly. 

As another example, GAO's discussion of the prime sponsors' difficulties 
in the rapid start-up of the program should be looked upon as a learning 
process in how to structure the initial implementation of a large-scale 
program, not as a criticism of the prime sponsors per se. In fact, the 
willingness and ability of several prime sponsors, such as Baltimore and 
Detroit, to readjust their management structures in the face of extreme 
initial difficulties could be looked upon as a virtue of the system r=.ther 
than a criticism of it. 

2 GAO's approach is further troubled by its failure to recognize 
sufficiently that a basic purpose cf the demonstration was to assess the 
impact of a school-conditioned job guarantee on the longer-term employment 
and educational experience of its participants. It is important to understand 
this facet of the demonstration's research for two reasons. Perceived 
problems with specific aspects of the program (i.e., limited counseling, 
lower-than-expected reenrollment of drop-outs, difficulties with educational 
standards) might not be of sufficient weight to require changes in the 
program's structure if the overall impact on the youths is significantly 
positive. in addition, to ensure a reasonable test of longer-term impacts, 
it was necessary to provide a sufficient level of supervision and control 
over a demonstration to assure that it had a fair chance of yielding the 
necessary information. While it is therefore true that a limited short- 
term demonstration can never be "real world" in the sense that I think GAO 
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is using the tenn, it still is surely one of the best techniques we 
possess to inform policy as to what works and what doesn't under reasonably 
realistic conditions . 

3. Another basic disagreement with the GAO evaluation is the way 
standards of performance are applied by GAO in its analysis. This issue 
becomes important because the report implies standards of quality against 
which the various facets of the program's operation are judged: GAO proceeds 
to find fault by citing negative examples when its implicit standards are 
not met. Therefore, because the tone of the report and the examples or 
illustrations used are heavily slanted toward the negative, the reader is 
left with a distinct impression of failure. 

The question should be asked: A failure compared to what? In a 
complex demonstration v^hich found jobs for over 80 r 000 youths in a relatively 
short period of time, one can, no doubt, find examples to justify any state- 
ment. A balanced analytic approach, however, would call for selecting 
examples not only in the context of overall program accomplishments, but 
would also ask what could reasonably be expected from an employment program 
geared to this population. For example, the report cites an ongoing 
problem that Tier I sites had in developing sufficient jobs for their parti^ 
cipants. While we agree that there were occasional difficulties with job 
development, on the whole the ability of prime sponsors to cr«ate rapidly 
large numbers of reasonably high quality jobs seems to have been a 
program strength rather than a program weakness. Or, as another example, 
the report appears to find fault with the prime sponsors' ability to work 
with the other community agencies and institutions and again provides 
examples to justify its findings. We vrould suggest that, on balance, the 
prime sponsors \/ere generally able to work out fairly effective relationships 
with numerous Iccal agencies and, in fact, many of the examples cited by 
the report as apparent prime sponsor failures were actually due to inadequate 
performances by i:he delegate agencies. These necessitated that prime sponsors 
take action to terminate or alter relationships in the interests of sound 
management . 

4. As our last major area of concern, we would point out several 
instances where the report deals less than adequately with issues that it 
raises- These include the following: 

a) Counseling - *nten5=:ive, individual counseling was not a 
demonstr=i tion objective. While some may argue thr^t 3uch counseling is 
needed, it is unfair to imply a failure to achieve this as a program goal 
when it was not part of the design. 

b) Program Management Costs and (Jnder-l^nrollme.'i t - Both of 
these terms as used in the report are misleading. Prograra management costs, 
as distinct from client services costs (as that term is non^ally used in 
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the CETA system), were not particularly high in this program, averaging 
less than 20% of total costs. We believe it is, therefore, not accurate 
to cite such costs as being high. Furthermore, to characterize partici- 
pant levels £is **under-enrollment" also implies under-achievement * In 
fact, participation levels (now approximately 50% of eligibiles) / while 
lower than site estimates, are higher than the researchers' predictions. 
Again, going back to the purposes of the demonstration, the fact is that 
no one knew what participant levels would be, and a basic purpose of the 
demonstration was to find this out. Once participant levels and manage- 
ment and service needs became clearer, it was logical to make adjustments 
based on experience and within the cost parameters generally acceptable 
in the CETA system. 

c) Displacement - This section of the report is particularly 
weak; it deals superficially with a very complex issue. First, the report 
states that £k\ in-depth review of the issues was not conducted, and then, 
citing a few negative examples, concludes that displacemerc or substitution 
occurred in 45% of the private sector jobs, based solely on the age of 
normal hires. While it is altogether possible that the report's figures 
will eventually prove accurate, it is clear that the level of a lalysis 
carried out in this study is not sufficient to justify such sweeping con- 
clusions. A major study of this issue, being carried out at the behest of 
the Department of Labor and the Office of Management and Budget, is 
currently underway, and we are hopeful that it will better infonn this 
subject, 

d) Adequacy of Data in the Information System - The report con- 
tinues tc confuse short-term data problems, uncovered by GAO investigators 
at the sites, with the ultimate reliability of the data in tho Information 
System used for research and reporting purposes. It also misinterprets 
quality control requirements, generally giving their negative implications 

either than reali2.ing that they are a means for t.-suring data completeness, 
accuracy and timeliness. Rather than go into this issue in detail in this 
letter, I am attaching a fuller exposition of it. 

As a final comment, I would note that the specific findings of GAO on 
the experiences of the Entitlement Demonstration are not substantially at 
variance with many of our own findings, which v;e have issued publicly in 
reports on the quality of work and the general implementation experience of 
the demonstration. Our view is that, as a learning experience which has and 
will continue to inform policymakers on what accomplishments are possible 
in this complex arena, the demonstration has been a positive achievement. 

Since:-ely 



William J. Grinker 
President 
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THE FNriT'.M'iKICT INrCPa'WfiOr! f^VSTEM 

With re3j...*ct to GAO*s cc.;r^tr:ts on tiho Entitler.c?nt Infcrm'i tion £y3t-.-.m 
(EIS) and site cat.i collect, i ...n ii.i r .ixvrting activities^ th» re are throe 
areas -lu-sctvi-vj of rlnrif i -a? i^n: the relationship of tae Rntitlcriont 
Irfo^n^ j-:ion Syi5t'::m to ^ht3 v..riojs ir :.f?-^rch ftn.Uyses, the rols^z.ionship of 
local data* systems to the MlRr I:iforr:at ion Fystem, r>.n.i the intorprt^tation 
nnd pivs^ntation of fact. 

is in: orti.int ro nake ^he distincti'^n between vh^t CAO viesciibes as 
•lata proM^ins lee^ultina from k; c.Msor lecoi vikc€*ping md reporting, and the 
accuracv and rol i ..bil ity of the dita finally used in inip,ict, cost and 
inplor.ontation research analysis. First, data provided by the sponsors have 
absolutely no connection with data collected independently by the research 
suJTiCOii tractor from the field survey efforts in experimental and comparison 
sites. The impact analysis is primarily dependent on the field survey data 
collection activity, and it is supplemented by program EIS data only after 
intensive data verification and cleaning efforts, Tlie Entitlement Information 
System is used extensively in the cost and implementation analyses, and 
continu.njs counter-checks and reconciliations have been employed to assure that 
the dat I used for analysis are of sufficiently high quality for research and 
reporting purposes. 

Pioiram dnta submissions originate from various local sponsor units and 
or- Kxibject to three levels of quality control by MDRC staff. First, each 
in. un \t f.nd report is rr*^ iewed independently for completeness an:\ validity 
. ..o* lo-,*:. Thi.^ first ^evel of qualiry control t,-ns>ures the g:;:iei*l 
3- 'A-.- i ility of tht dociirTiOnts into the computerised infoiir.ation system., 
.nec: ;^ -lilcviri.i cr^Tmuter pLC>ccssi:iq. errors relating tc a form's locric in 
sec .-i--^''- c.r conto:it with leipt^ct to other documents of a similar ty e are 
resolvwd. This is done by -arefui review of the source doco-r.ert, by telephone, 
/->»- b" : 1 bui i o!» of :\/Uj: •■itc^r-q<^ncrated error leportK to the r:;:^onsor. Third, 
an i rjir.i V- rec c:t.'i 2 lot ion ; taVes pla'X-, invo^'ing nor vr.]y t::e Mrs 



ntc b,:t "ils' 'rl-' .mkI -t u I ef-hone call-in ^^.ata. Tiis \'-:o':'.:U'ii 



C'.i^ : J ''.^ tb^.t v.rri" 11;: w> .-^y .w" 'iritu - cmi roi r i- *nt , stnt.:5, wuqes an^. l^o*.; ^;«woj. >;ed 
- .ire 3-'ir*callv w': i M-cr't , b^Mih for an i:^dividual pc^rt icipant ar.S ir. the 
cngro^.»te, rer:aral'«^s rf wliich iu.>i:r.;m unit prepared the data and w}a -h '/ype 
of do- ximent cont.=iirii'.-d the infc.rmal ion. Discrepancies found throu,ih the '^jality 
4-ontrol processes .iie r« solved by repeated contacts with local oper itors nnd, 
if iiocded, on-site viniln. All CIS data r^J'^^sonted in MDRC rosearch reports will 
aave crnno through these levels of quality control and will have been fullv 
recojiciled. 

It is also importont to understand the differerice between the centraliy.ed 
liifonnation Syritcni ond 17 possibly unique locnl systems. Prior to Entitlomont, 
./ich pritoo sponsor had an t-xisting information system, with local forms, 
^systems and procedures. Entitlement imjx^sed a second set of foims and procedures 
Although prime s:vinsors were encouraged to assimilate these two systems and to 
integrate procrdures as much as possible, many sites decided to retain both 
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-yf,r...n-j .:f. i...-. o oi.tiM^s. \:hcn this di-! -.u.-cur , aiC-.Mer prioiity v.is 
ur....-vny rl .w.vi on n.i i-. ' i nliij M.e .ccnracy of the Iccal system, f..),'-:n.s, .md 
fi: _-s. MORC cr>nt iinjoiisly toviewod ind rcconcl 1 c-d past ^lata but the priiue 
rspo'i-ors were cc-n.: l oliy' ;:iwre concpined with keeping thoir c-irrer.t files 
.icciirate. r-.u ri-,.-.:-,,, although all nodi f i •v.i r ions resulting from d.^ta 

reconciliations v.-. re piovided to local sitos, they may or may not have been 
.■jpplied to the V>~cil ri.-na isystems, .ler-c-ndinq upon whether the Ppec-ific MDHC 
forxs wore utUi-'.-d 1 .-ally nnd wh-vther ;he data »-olated ^.o current or 
historical ..ctivit.y. This situation could o.-ssily t.ave resulted in ri.jnifivant 
Vv»>-i.-«:.-e bt:tw.--r'r. 'he ^i.-.-ility of MrftC's data -jnd th.jt of the Mriir.c sv-r.isors. 
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CITY OF ALBUQUERQUE-BERNALIULD COUNTY 
□ FFICE OF COMPREHENSIVE 
EMPLOYMENT AND TRAINING 
ADMINISTRATION 



302 SIKTM ST.. S.W 
ALBUOUEROUE. NE.W MEXIflO R7in2 



September 26 



1980 



Mr. James Walsh, Group Director 
United States General Accounting Office 
Human Resources Division 
Washington D^C. 20548 

Dear Mr* Walsh: 

As requested we have reviewed the proposed report 
to Congress on the Youth Incentive Entitlement Pilot 
Project . 

The report is a fair representation of Y.I.E.P.P. 
The favorable co^rdr.ents about the Albuquerque project 
are appreciated and we do not have any other comments 
regarding the report* 



Sinoerel 




CD/LC 



CC: Central File 
Or igi nator 
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CITY or BALTIMORE 

WILLIAM nONALl:) bCHAEFF.R. Moyor 




M.\YOR'S OFFICE 

OF MANPOWER RESOURCES 

MARION W. PINES. Director 

701 St. Paul Street. Suite 500. Ballimore, Maryland 21202 



September 22, 1980 



Gregory J. Ahart, Director 

Jnited States General Accounting Office 

Washington, D.C. 20548 

Dear Mr* Ahart : 

. ^ 4. 4.U A^^p^ r^f voiir renort on the Youth Entitlement 

as we analyze the implementation and impact of Entitlement. 

1. The Demonstration was. in part, designed t'^^^^^^^'^i^^.f iJ^^^^ 
were indeed large numbers of unemployed youth who would be wiiimg 
to work if they had the opportunity. 

2. The Demonstration also sought to determine ^^ether a significant 
involvement of the private sector could be enlisted m a jobs 
prorirajn for youth. 

^ It also sbuPht to determine whether Prime Sponsors could develop 
Jhe ianageS^ capacity to operate such large scale employment 
programs . 

perhaps the most significant fact about the Entitlement Demonstration is 

A„„th.. factor „hioh »u.t be kept in -ind "^-/'.--^i'Jf .rro^frirtE'lii" 
th.t. When a l.rg. eit, n.^ a^ray's un^.r.tand ^he oo.pl.xity 

becomes widely publicized. The general P"°^ public and political presE,ure to 

involved, or the need for ' ^-''^^"^ ^^^^"^^1; do wt have tSe luxury of taking 

••pet started" is enormous. Seldom in such a case qo we 
the time really needed for optimum program implementation. 

■ P ^he reneral Accounting Office, I was told by several 
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provide whatever was requested. Some mistakes have been made» and we were 
willing to face up to those mistakes , knowing that we had already recognized 
most of them and had already moved to effect change. I fear, however, that 
this policy of openness may have boomeranged- Let me be more specific. 

1. The report is quite unbalanced. First, your technique of stating 
a general conclusion and supporting it through the use of anecdotal 
instances from one or two sites rather than specific data, invites 
unwarranted generalizations. Secondly, all sites had both positive 
and negative experiences. Care should be taken that balance is 
maintained in your selection of examples, and that no site is picked 
out for only negative examples. This also invites unfavorable con- 
clusions • 

2* Tlie report does not recognize the value of change. A Research and 
Demonstration Project, in a review such as GAO conducted, must be 
viewed differently from an ongoing, operational program. Change 
should not be considered a negative thing. Program operators in 
a Demonstrative Project have a responsibility to make changes and 
adjustments in operational systems in order to develop the best 
possible administrative structure for future replication. We, there- 
fore, suggest that creative response to the need, for change ought to 
be viewed as praiseworthy rather than be subject to criticism. 

Similarly, while criticism of poorly functioning systems is certainly 
valid, when those systems have been replaced or modified nakino a 
significant improvement in management operations, such should at least 
be recognized in the report. 

3. A number of factual or interpretational errors exist which should be 

corrected. For example, you state on page 21 that negative terminations 
exceed positive ones. Yet the only positive termination you recognize 
is '^high school graduate". In every other CETA Program, placement in an 
unsubsidized job is considered a positive outcome. If you add the 11,7% 
of job placements to the 26.7% of high school graduates, the total positive 
ouL^orac lalc lb 30. ^To, which exceeds* the 34*1% ncgaLive tcrmii\alions • 
Presumably there are other terminations classed as "neutral" which are in 
reality positive as well. 

Another example is on page 30* While noting that Baltimore has the lowest 
cost per participant, you attribute that to a lower number of hours worktd 
by participants in the Baltimore program. However, if adjustment is made 
for the lower hours, taking into account attendance rates and breaks, 
Baltimore still has the lowest cost per participant. 

A. A governmental agency preparing a report for Congress should be extremely 
cautions of how it handles quotes and materials taken out of context. On 
page 45, in quoting MDRC sources regarding the Baltimore program, material 




GAO note: Page references have been changed to agree with 
the final report. 
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is taken from several reports issued at different times is removed 
frol seJerardifferent contexts, misquoted and rearranged so as to 
convey a "tally different impression from that which MDRC intended 
in any of the reports. (For example, the "98% error rate for forms 
noted in this section refers to one week in July, during a peak 
payroll period, and was not. as one could imply, a normal condition.) 
Se suggest that all such quotations and paraphrases be ^-'^^'^^d and 
adjusIL to more clearly and objectively transmit the author's intent. 

in conclusion, we are pleased GAO found that the vast -jority /;;title^-"^ 
worksites fostereH pood work habUs. provided ^ ^ ^ « ^^"^ ' ^^"'eLus i n 

and orovides adequate supervision for program participants. With such -^esuics in 
mi^d U cannot Se doubted that the majority of program participants -^^^^^^^ 
ZTuL^il and meaningful experiences during their tenure in the program^ to .ap^ y 
stand how very difficult it must have been for your field staff to seek to rapiaiy 
assimilate digest the vast amounts of information collected regarding this 

:rrrJim !er program. We attempted to give -r c tion in his ort.^^^ 

and we hope that our comments "P- your first ^-f^ ^J the report wi be^^^^ 
to you in further refining your analysis. We sincerely nope 
not simply a warm-up for your next presentation on "60 minutes. 

Sincerely , 'J ' 

Marion W . Pines 
Director 



MWP : sk 
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Southern New Hampshire Services, Inc. 



EMPLOYMENT AND TRAINING PROGRAM 



P,0, Box 5040 
Manchester, New Hampshire 031 08 
Telephone: 603-669-0701 



JAMES A. MACHAKOS 
CETA Administntor 



Office: Route 114 

Go'ffslown. NH 03045 



September 15, 1980 



Mr. Gregory J. Ahart 
Director, Human Resources Division 
United States General Accounting Office 
Washington, D.C. 20548 

Dear Mr. Ahart, 

Thank you very much for the opportunity to review the proposed 
report to the Congress on the Department of Labor's Youth Incentive 
Entitletnent Pilot Projects and to comment on the material in the 
report before it is issued in final form. 

At this time the Hillsborough County Prime Sponsor will not be 
commenting on the proposed report and is returning both drafts to 
your office. 




JAM/ It 
Enclosures 
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CITY OF PHILADELPHIA 



OFFICE OF EMPLOYMENT AND TRAINING 
1234 Market Street. 3rd Floor 
Philadelphia. Pa I9l07 



WILLIAM J GREEN 



WILLIE F. JOHNSON 



MAYOR 



September 24» 1980 



Director of Human Resources 

Division 
U.S. General Accounting Office 
441 G Street, N.W. 
Washington, D.C. 20548 

^ear Mr- Aharg: 

Thark you very much for the opportunity to comment 
on the draft of GAO * s report on the Youth Incentive 
Entitlement Pilot Project. Philadelphia's experience, 
generally, has been complex and variable but specific 
components of our Entitlement have shown considerable 
merit and deserve replication. Among them are: 

1. Private sector worksite development; 

2. Clerical Skills Enrichment; and 

3. Concentration of services in a targeted area. 

Although these components may not be duplicated 
exactly, variations are planned for future programs. 

Ur.f orcunacely , there was uo mention of these in 
the draft report. We were in general agreement with the 
major findings in the areas of operational problems, 
worksite experiences, and the test of YIEPP as a guaranteed 
job program. 

Regarding the Issue of the frequent intervention of 
Labor and its contractor in ^program operations, we are 
of the opinion that the causes are worthy of further study. 
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Existing CETA delivery syst.^ms operate without inter- 
mediaries such as Manpower Demonstration Research Corporation, 
Many operational and administrative constraints that have 
occurred in the Philadelphia Entitlement Program can be 
directly attributed to this relationship and they do not 
occur in our relationship with the Regional DOL office. 

We urge you to seriously re-evaluate the content of 
the draft report with intent to include a section on MDB.C ' s 
role as the program manager - 

Thank you very much for your consideration. 




James Rowley 
Deputy Director of 
Operations 

JR:AH:ar 
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STATE OF MISSISSIPPI 




omcc or tmc 



DICK MOLPUS 



DP ROBERT L. ROBINSON 



COORDINATOR 



EXECUTIVE DIRECTOR 



FEDERAL • STATE PROGRAMS 



JOB DEVELOPMENT & TRAINING 



September 26, 1930 



Mr. Gregory J. Ahart, Diriictor 
Human Resources Division 
United States Accoionting Office 
Washington, D.C. 20548 

Deax Mr. rJiart: 

Thcink you for providing us the opportunity to comment on the draft 
report to the Congress on the Department of Labor's Youth Incentive 
E.icitlement Pilot Project. Hiis office operates one of the largest Y I EPP 
programs in the nation and we certainly feel that it has been the most 
successful. 

Staff has carefully reviewed and analyzed this draft ar. 1 concurs 
with the major findings. In addition, we fully support the recommendations 
detailed on pages 53 and 54 for the same ones have been made at one time 
or another by our monitoring cmd program staff. However, a few 
discrepancies or areas needing further clarification were noted. These 
are listed below: 

- The report revealed a need for additional monitoring as 
cited on page 22 and included Mississippi in the list of 
sites using a monitoring checklist. Perhacs the GAO team 
did not review our monitoring reports for they are very 
detailed in nature and are not based on single checklists. 
Monitoring as conducted by the Governor's office of Job 
Development asid Training was and remains thorough and 
complies with regulatory requirements. 

- The statement on page 31* " sponsors were 

reluctant to terminate youth and were slow to enforce 
standards," is misleading in regard to our program. 
Clearly defined academic standards for In-School or 
Alternative Ekiucation students and class attendance 
standards for Alternative Education students were not 
established until the last s±y months of enti';lement. 
However, at that point, the r.Ccindards were enforced 
and participants who did noc meet such standards were 
terminated. 



P, O. BOX 22808 • JACKSONi MS 39205 • (60? J 'J54-7676 
"An Equal Opportunity Employer M/F/H" 
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- The report states that, "Sponsors placed a low priority 
on counseling." In reality, the In-School Entitlement 
Counselors were extremely restricted on the amount of 
time which could be spent with participants for the 
participants were either in class or working. More 
counseling is, indeed , needed cind these difficulties 
should be considered in designing ciny related program. 

Again, we af>preciate this opportunity and your consideration of our 
comments. If you have any questions, please contact Mr. Herman Wells at 
601-354-6155. 




Robert L. Robinson 
Executive Director 



RLR/JD/mg 



6 AO note: 
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U. S. Department of Labor tnspeclor General 

Washington. D.C. 20210 




OCT 9 1980 



Mr. Gregory J- Ahart 
Director 

Human Resources Division 

U.S. General Accounting Office 

Washington, D.C. 20548 

Dear Mr. Ahart: 

This is in reply to your letter to Secretary Ray Marshall 
requesting conmients on the draft GAO report entitled, 
"A Major Department of Labor Demonstration Program Provides 
a Number of T.es Jons on Implementing Yoxith Programs." The 
Department's r'lisponse is enclosed. 

The Department appreciates the opportunity to coioment on 
this report. 




RC»NA1.D GOLDSTOCK 

Acting Inspector General 



Enclosure 
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U,S, DEPARTMENT OF LABOR'S RESPONSE TO 
THE DRAFT GENERAL ACCOUNTING OFFICE REPORT 
ENTITLED — 



11 



A Maj or Department of Labor 
Demonstration Program Provides 
a Number of Lessons on Implementing 
Youth Programs " 



Recommendation: 



T^*c Secretary should require sponsors to strictly enforce 
reasonab3<i attendance and academic standards thei.eby enhancing 
the program's credibility and effectiveness for both participants 
and the general public. 



Response ; 

The Department concurs. 



Comment r ; 

This requirement has been demanding in t-.erms of time and effort 
during the demonstration. There has been a dis juncture between 
the monthly report requirements called for in the demonstration 
and the rePor'j;ing patterns that exist in the various school systems, 
as well as the problems of insufficient personnel to deal with both 
data and stucie^. _ needs on a timely basis. Although lengthy 
negotiations between the school and prime sponsors were required 
Tco establish the standards in the first place, all parties hav- 
com-5 to believe that standards and attention to enforcement could 
serve the youth and th . program better. However, tin:e and 
increasing cooperation between schools and prime sponsors 
suggest that the enforcement of standards is feapiible. 



Recc ; nmenda tion ; 

The Secretary should increase efforts to promote cooperative 
relarioTiships between sponsors, schools and other organizations 
serving youth. 



Response : 

The Df^Tpar tment concurs . 
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Comments : 



These cooperative relationships should revolve around the 
appropriate functions of the organizations. The Entitlement 
Demonstration has shown that the various agencies did best 
what they had done well before. In addition, the cooperative 
relationships can best be maintainec* through a strong monitoring 
effort on the part of the Prime Sponsor to ensure that the 
guidelines are enforced and coordination occurs. 

Recommendation : 



The Secretary of Labor should assirl Prime Sponsors in upgrading 
worksite supervision, job development, participant counseling, 
and monitoring. 

R esponse : 

The Department concurs. 



C ommen t 

Should the Demonstration be continued. Prime Sponsors will 
establish expectations for the v/orkplace including the 
supervision, standards, ?ob content, clarity of the assignment, 
and value* The supervisor will be required to be available, to 
answe^: task relate.i questions, provide continuing on-the-job 
training, communicate perforinance expectations, and provide 
positive reinforcement for exemplary work. The suo'-r isor 
will establish worksite standards and communicate uo the 
participants that adherence to standards is a condition of 
continuing participation. Although there are inherent limits 
to the skil3. requirements which can be placed on youth who 
have not yet completed high school and who may have need of 
remedial education, job tasks and the stanc'ards for those 
tasks should meet the capabi litie'.J of the participants. When 
the tasks to be completed and the productivity expected are 
clearly defined, it is easier for participants to know when 
they are working well, when they are meeting the expectations 
of their supervisors, and when they have accomplished their 
tasks* In addition, the participants are more likely to 
perform better if they believe that the \/crk makes a 
contribution to the agency's goals and objectives. Counseling 
activities will be upgraded as the demonstration phase of 
Entitlement comes to an end because as simi^.ar programs are 
adopted in the future, they will not have the size or magnitude 
of the demonstration. The GAO report implies that participants 
received more frequent and in-depth counseling at smaller 
sized programs. 
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Recommendati on : 

The Secretary of Labor should create, implement, and enforce 
adequate controls to prevent substitution in the workplace. 



Respons e : 

The Department concurs. 



Comments : 

Although the true displacement effect will not be known 
until the report is completed in April 1981, steps can be 
taken now to assure that minimal substitution patterns 
develop when using private sector businesses as worksites. 
The first method to curtail substitution will be to limit 
the number of private sector worksites that normally hire 
youth without the subsidy such as fast food establishments. 
With the requirements of the proposed legislation automatically 
limiting the size ot Entitlement programs in the future it 
will be easier to develop an adequate number of private 
sector jobs that are not generally reserved for the youth 
population. In addition, the establishment of and adherence 
to a reduced subsidy level plan as a condition for continuing 
participation in Entitlement will minimize the substitution 
problem. 



Recommendation ; 

The Secretary of Labor should establish and enforce requirements 
that limit program management costs. 



Response : 

The Department concurs. 



Comments ; 

In an ongoing operational mode , it would be easier to establish 
acceptable management cost levels because there is now some uata 
from which to draw information. Originally, the major cause of 
the high management cost rates was the underen rol Iment of 
participants. Experience from the demonstration will permit a 
more appropriate estimate of the take up rate for participation 
and sponsors will better judge the impact of competition from 
other youth programs. 
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